Add support for 'release-debug' build for ATDM Trilinos builds and use to avoid timeouts
Created by: bartlettroscoe
CC: @fryeguy52, @mhoemmen, @rppawlo, @bathmatt, @micahahoward, @trilinos/kokkos, @trilinos/kokkos-kernels
Next Action Status
ATDM Trilinos scripts now support a
release-debug build type and this has been used in new
release-debug builds on 'waterman'. Converting
debug builds to
release-debug builds on other platforms will be done in follow-on issues ...
Currently, the ATDM Trilinos builds support
debug and an
opt build. The
debug build uses
-O0) and enables runtime debug-mode checking while the
opt build uses
-O3) and no runtime debug-mode checking. The problem with this approach is that some of the Trilinos tests (especially many of the Kokkos and KokkosKernels tests) run many times slower wtih
-O0 than with
-O3. This has caused many tests to timeout at 10 minutes in
debug builds that finish is well under 10 minutes in
opt builds (e.g. #2964, #2921 (closed), #2461).
A solution that we discussed was to change most
debug builds into
release-debug builds that will set
-O3) but enable runtime debug-mode checking.
The idea would be to add a new
release-debug keyword that matches before
debug which will set
ATDM_CONFIG_BUILD_TYPE=RELEASE_DEBUG and then update the file
ATDMDevEnvSettings.cmake accordingly. That will be easy. The harder part will be updating the tweaks
*.cmake files and all of the Jenkins jobs to accommodate the name change. NOTE: calling this
release-debug as apposed to
opt-debug hopefully might be more clear.
We can still leave some full
debug builds to help support full GDB debugging by the ATDM APPs teams but they should be sparing (because we are constantly dealing with timeouts with full debug builds).