SEACASExodus tests failing in new cuda 9.2 ATDM build on white/ride
Created by: fryeguy52
CC: @trilinos/seacas , @kddevin (Trilinos Data Services Product Lead), @gsjaardema , @bartlettroscoe
Next Action Status
PR #3418 which switched from module netcdf-exo/4.6.1/openmpi/3.1.0/gcc/7.2.0/cuda/9.2.88 to netcdf/4.6.1/openmpi/3.1.0/gcc/7.2.0/cuda/9.2.88 on white/ride merged on 9/10/2018 and SEACAS tests fully passed on 9/11/2018.
Description
As shown in this query the tests:
- SEACASExodus_exodus_unit_tests
- SEACASExodus_exodus_unit_tests_nc5_env
are failing in the builds:
- Trilinos-atdm-white-ride-cuda-9.2-opt
- Trilinos-atdm-white-ride-cuda-9.2-debug
The output looks similar to what we were seeing in #2815 (closed)
================================================================================
TEST_0
Running: "/bin/bash" "/home/jenkins/white/workspace/Trilinos-atdm-white-ride-cuda-9.2-opt/SRC_AND_BUILD/BUILD/packages/seacas/libraries/exodus/test/testall" "netcdf5"
Writing output to file "/home/jenkins/white/workspace/Trilinos-atdm-white-ride-cuda-9.2-opt/SRC_AND_BUILD/BUILD/packages/seacas/libraries/exodus/test/exodus_unit_tests.out"
--------------------------------------------------------------------------------
testwt - single precision write test...
Exodus Library Warning/Error: [ex_put_name]
ERROR: element block id 10 not found in file id 65536
testrd - single precision read test......
--------------------------------------------------------------------------------
TEST_1: Return code = 1
TEST_1: Pass criteria = Zero return code [FAILED]
TEST_1: Result = FAILED
================================================================================
OVERALL FINAL RESULT: TEST FAILED (SEACASExodus_exodus_unit_tests_nc5_env)in that issue SEACASExodus_exodus_unit_tests was also failing on mutrino and we ended up disabling the test for that build.
Steps to Reproduce
One should be able to reproduce this failure on the machine white as described in:
More specifically, the commands given for the system white are provided at:
The exact commands to reproduce this issue should be:
$ cd <some_build_dir>/
$ source $TRILINOS_DIR/cmake/std/atdm/load-env.sh cuda-9.2-opt
$ cmake \
-GNinja \
-DTrilinos_CONFIGURE_OPTIONS_FILE:STRING=cmake/std/atdm/ATDMDevEnv.cmake \
-DTrilinos_ENABLE_TESTS=ON -DTrilinos_ENABLE_SEACAS=ON \
$TRILINOS_DIR
$ make NP=16
$ bsub -x -Is -q rhel7F -n 16 ctest -j16