Belos test failures on CUDA and OpenMP (White/Ride)
Created by: crtrott
Last Integration of Kokkos/KokkosKernels into Trilinos still had quite a few test failures (see pull request #1721):
Cuda:
720 - Belos_pseudo_gmres_hb_MPI_4 (Failed)
721 - Belos_bl_pgmres_hb_0_MPI_4 (Failed)
722 - Belos_bl_pgmres_hb_1_MPI_4 (Failed)
723 - Belos_bl_pgmres_hb_2_MPI_4 (Failed)
724 - Belos_pseudo_pgmres_hb_MPI_4 (Failed)
725 - Belos_bl_fgmres_hb_MPI_4 (Failed)
741 - Belos_gcrodr_hb_MPI_4 (Failed)
743 - Belos_prec_gcrodr_hb_1_MPI_4 (Failed)
744 - Belos_rcg_hb_MPI_4 (Timeout)
OpenMP:
715 - Belos_pseudo_gmres_hb_MPI_4 (Failed)
716 - Belos_bl_pgmres_hb_0_MPI_4 (Failed)
717 - Belos_bl_pgmres_hb_1_MPI_4 (Failed)
718 - Belos_bl_pgmres_hb_2_MPI_4 (Failed)
719 - Belos_pseudo_pgmres_hb_MPI_4 (Failed)
720 - Belos_bl_fgmres_hb_MPI_4 (Failed)
736 - Belos_gcrodr_hb_MPI_4 (Failed)
738 - Belos_prec_gcrodr_hb_1_MPI_4 (Failed)
Check out the kokkos-kernels/scripts/trilinos-integration/white_run... scripts for how to run this.
The gist is you need to set this and run the below configure script:
module load devpack/openmpi/1.10.4/gcc/5.4.0/cuda/8.0.44
export OMP_NUM_THREADS=8
export JENKINS_DO_CUDA=OFF
export JENKINS_DO_OPENMP=ON
export JENKINS_DO_PTHREAD=OFF
export JENKINS_DO_SERIAL=OFF
export JENKINS_DO_COMPLEX=OFF
export JENKINS_ARCH_CXX_FLAG="-mcpu=power8"
export JENKINS_ARCH_C_FLAG="-mcpu=power8"
export BLAS_LIBRARIES="${BLAS_ROOT}/lib/libblas.a;gfortran;gomp"
export LAPACK_LIBRARIES="${LAPACK_ROOT}/lib/liblapack.a;gfortran;gomp"
export JENKINS_DO_TESTS=ON
export JENKINS_DO_EXAMPLES=ON
export QUEUE=rhel7F
module load python
sampleScripts/Sandia-SEMS/configure-testbeds-jenkins-all