Skip to content

Swap out OpenMPI 3.1.0 for OpenMPI 2.1.2 on white/ride (#3290)

James Willenbring requested to merge bartlettroscoe:3290-white-ride-use-openmpi-2.1.2 into develop

Created by: bartlettroscoe

@trilinos/

Description

Here, just the MPI implementation is swapped out but the TPLs for OpenMPI 3.1.0 remain (because they have not been built for the GCC 7.2.0 + OpenMPI 2.1.2 + CUDA 9.2). It is not clear how this can work but it is all that we can do without the TPLs properly built for this env.

I also switched to the GCC 7.2.0 + OpenMPI 2.1.2 + CUDA 9.2 env for the GNU non-CUDA build just to be consistent.

Motivation and Context

We want the Ifpack2_BlockTriDiContainerUnitAndPerfTests_MPI_4 test failure to go away described in #3290 (closed). It looks like OpenMPI 3.1.0 is not doing the right thing with CUDA memory but OpenMPI 2.1.2 appears to be.

How Has This Been Tested?

On 'white' I ran:

$ bsub -x -Is -n 20 ./checkin-test-atdm.sh  all \
  --enable-packages=Kokkos,Ifpack2,SEACAS,Panzer --local-do-all

and all of the builds and tests passed (including the Ifpack2_BlockTriDiContainerUnitAndPerfTests_MPI_4 test). I picked these package because of their usage of CUDA and TPLs. And Panzer in some way tests most of Trilinos (at least somewhat).

Checklist

  • My commit messages mention the appropriate GitHub issue numbers.

Merge request reports