Skip to content

Add install hooks for ATDM Trilinos configuration and ctest -S driver

Created by: bartlettroscoe

CC: @fryeguy52

Description

In order for the ATDM projects to adopt the new ATDM Trilinos Configuration and Jenkins Drivers (that submit to CDash as a byproduct), these scripts must support the installation of Trilinos and using Trilinos from that install.

For this to occur:

  • There must be a single that can be sourced sitting in the installation directory that will load the correct env to use that installed version of Trilinos. [Done]
  • Trilinos must be installeable using that ATDM Trilinos configuration from a Jenkins job

Possible solutions

First, a new env var ATDM_CONFIG_TRIL_CMAKE_INSTALL_PREFIX could be added that would be read in the ATDMDevEnvSettings.cmake file and would set the CMake var CMAKE_INSTALL_PREFIX. That would give you the correct configuration of Trilinos and would know where to install.

As for doing the installation itself, one simple solution would be to provide a atdm/install-prebuilt-trilinos.sh script that will run make install on the built Trilinos version. This script would need to be explicitly called by some Jenkins driver script. But any failures would not be posted to CDash.

If we wanted to allow for installation errors to get reported up to CDash, then a CTEST_DO_INSTALL option could be added to the TRIBITS_CTEST_DRIVER() function and would do a cmake_build( ... ) command that would run the install target (and in parallel) after the project build was complete. That would also allow logic for not installing if there was a build failure and, again, would allow for any install failures to get reported to CDash.

Also, to help to make installation testing for downstream customer codes stronger, the smart-jenkins-driver.sh script could be updated to move the source and build directories out of the way to catch mistakes were the installed configuration files point into the source or build trees. This could be accomplished by adding the steps:

  • Run the smart-jenkins-driver.sh script under $WORKSPACE/Trilinos
  • If $WORKSPACE/moved/SRC_AND_BUILD/ already exists, the move it back to $WORKSPACE/SRC_AND_BUILD/
  • After the build, installation and testing are completed, move $WORKSPACE/SRC_AND_BUILD/ to $WORKSPACE/moved/SRC_AND_BUILD/.

NOTE: The reason to move $WORKSPACE/SRC_AND_BUILD/ to $WORKSPACE/moved/SRC_AND_BUILD/ instead of say $WORKSPACE/SRC_AND_BUILD.moved/ is to catch errors where the installation under $WORKSPACE/local_install/ might have relative paths to $WORKSPACE/SRC_AND_BUILD/BUILD/ which would also work for $WORKSPACE/SRC_AND_BUILD.moved/BUILD/ but would not work for $WORKSPACE/moved/SRC_AND_BUILD/BUILD/. Also, the reason to move $WORKSPACE/SRC_AND_BUILD/ instead of deleting it is to avoid having to re-clone the Trilinos git repo again from scratch every time the job runs and to allow looking through the build artifacts on Jenkins after the job is complete.

Lastly, in order to support loading the correct env from a Trilinos install, the relevant files from Trilinos/cmake/std/atdm/ need to be installed that match the configuration of Trilinos being installed. At a minimum, this must include, for example:

  • Trilinos/cmake/std/atdm/load-dev.sh
  • Trilinos/cmake/std/atdm/utils/
  • Trilinos/cmake/std/atdm/<system-name>/environment.sh

But that would allow loading any env, including those that don't match the install. So the Trilinos install hooks should also be updated to install a script <install-prefix>/load_matching_env.sh that takes no arguments and will source the installed atdm/load-env.sh script with the right job name. For example, this installed script load_matching_env.sh might look like:

source <install-prefix>/share/atdm-trilinos/load-env.sh Trilinos-cuda-9.2-opt

The a client like EMPIRE or SPARC would just source:

source <trilinos-install-prefix>/load_matching_env.sh

and then inspect the exported ATDM_CONFIG_* vars and know that the right compilers, MPI, TPL locations, etc. were loaded to correctly use that installation of Trilinos.