Belos tests timing out on ATDM intel-18 mpich build

Created by: fryeguy52

CC: @trilinos/belos, @srajama1 (Trilinos Linear Solvers Product Lead), @bartlettroscoe, @fryeguy52

Next Action Status

Description

As shown in the links below the tests:

• Belos_rcg_hb_MPI_4
• Belos_gcrodr_hb_MPI_4

are randomly timing out in the build:

• Trilinos-atdm-cee-rhel6_intel-18.0.2_mpich2-3.2_openmp_static_opt

Current Status on CDash

The current status of the Belos tests on this build for the current testing day can be found here

Steps to Reproduce

One should be able to reproduce a build where this is randomly failing on a machine with a cee rhel6 environment as described in:

• https://github.com/trilinos/Trilinos/blob/develop/cmake/std/atdm/README.md

More specifically, the commands given for a machine with a cee rhel6 environment are provided at:

• https://github.com/trilinos/Trilinos/blob/develop/cmake/std/atdm/README.md#cee-rhel6-environment

The exact commands to reproduce the build where this issue is randomly occurring should be:

$ cd <some_build_dir>/
$ source $TRILINOS_DIR/cmake/std/atdm/load-env.sh Trilinos-atdm-cee-rhel6_intel-18.0.2_mpich2-3.2_openmp_static_opt
$ cmake \
 -GNinja \
 -DTrilinos_CONFIGURE_OPTIONS_FILE:STRING=cmake/std/atdm/ATDMDevEnv.cmake \
 -DTrilinos_ENABLE_TESTS=ON -DTrilinos_ENABLE_Belos=ON \
 $TRILINOS_DIR
$ make NP=16
$ ctest -j16