SEACAS tests failing in ATDM CUDA builds starting 4/26/2018 (#2650) · Issues · James Willenbring / Trilinos

SEACAS tests failing in ATDM CUDA builds starting 4/26/2018

Created by: bartlettroscoe

CC: @trilinos/seacas, @gsjaardema, @fryeguy52

Next Action Status

Updated SEACAS is also causing mesh reading problems on non-CUDA builds for larger numbers of MPI ranks. PR #2653 was merged on 4/27/2018. which reverts PR #2625 updating SEACAS. New issue will be opened if next SEACAS snapshot cause an error.

Description

As shown in the query:

https://testing-vm.sandia.gov/cdash/queryTests.php?project=Trilinos&date=2018-04-26&filtercombine=and&filtercombine=and&filtercombine=and&filtercount=4&showfilters=1&filtercombine=and&field1=buildname&compare1=65&value1=Trilinos-atdm-&field2=status&compare2=62&value2=passed&field3=status&compare3=62&value3=notrun&field4=testname&compare4=65&value4=SEACAS

The SEACAS tests:

SEACASIoss_exodus32_to_exodus32
SEACASIoss_exodus32_to_exodus32_pnetcd
SEACASIoss_exodus32_to_exodus64

are failling in all of the current ATDM Trilinos CUDA builds:

Trilinos-atdm-hansen-shiller-cuda-debug
Trilinos-atdm-hansen-shiller-cuda-opt
Trilinos-atdm-white-ride-cuda-debug
Trilinos-atdm-white-ride-cuda-opt

This was likely due to the update of SEACAS into Trilinos in the commit 89d48adf merged in the PR #2625 .

Steps to Reproduce

One should be able to reproduce these failing tests on the machines white (SON), ride (SRN), hansen (SON), or shiller (SRN) as described in:

https://github.com/trilinos/Trilinos/blob/develop/cmake/std/atdm/README.md

For example, on white one should be able to reproduce these failing tests with:

$ cd <some_build_dir>/

$ source $TRILINOS_DIR/cmake/std/atdm/load-env.sh cuda-debug

$ cmake \
  -GNinja \
  -DTrilinos_CONFIGURE_OPTIONS_FILE:STRING=cmake/std/atdm/ATDMDevEnv.cmake \
  -DTrilinos_ENABLE_TESTS=ON -DTrilinos_ENABLE_SEACAS=ON \
  $TRILINOS_DIR

$ make NP=16

$ bsub -x -Is -q rhel7F -n 16 ctest -j16