Configuration of CUDA-enabled build fails
Created by: pelesh
Configuration of a CUDA-enabled Tpetra/Kokkos build fails at the point where CMake checks for C++ compiler. The error message is:
Linking CXX executable cmTC_babeb
/usr/tce/packages/cmake/cmake-3.5.2/bin/cmake -E cmake_link_script
CMakeFiles/cmTC_babeb.dir/link.txt --verbose=1
/usr/tce/packages/openmpi/openmpi-2.0.0-gcc-6.1.0/bin/mpicxx
CMakeFiles/cmTC_babeb.dir/testCXXCompiler.cxx.o -o cmTC_babeb -rdynamic
nvcc fatal : Unknown option 'rdynamic,-fexceptions,-pthread'
gmake[1]: *** [cmTC_babeb] Error 1
It seems as if Kokkos' nvc_wrapper
does not insert -Xlinker
flags correctly.
I am building from current master branch on a system with following configuration:
$ module list
Currently Loaded Modules:
1) StdEnv 2) gcc/6.1.0 3) cuda/9.1.85 4) openmpi/2.0.0 5) cmake/3.5.2
The same issue occurs with different openmpi, CUDA, CMake and gcc versions.
I am following Tpetra build instructions from $Trilinos/packages/tpetra/doc/FAQ.txt
, and I set environment variables like this:
export OMPI_CXX=${Trilinos}/packages/kokkos/bin/nvcc_wrapper
export CUDA_LAUNCH_BLOCKING=1
export CUDA_MANAGED_FORCE_DEVICE_ALLOC=1
I use following configuration script to set cmake options:
#!/bin/bash
cmake \
-D CMAKE_C_COMPILER="mpicc" \
-D CMAKE_CXX_COMPILER="mpicxx" \
-D CMAKE_Fortran_COMPILER="mpif90" \
\
-D Trilinos_CXX11_FLAGS="-std=c++11 --expt-extended-lambda" \
\
-D Trilinos_ENABLE_OpenMP=OFF \
-D TPL_ENABLE_Pthread=OFF \
\
-D Trilinos_ENABLE_Teuchos=ON \
-D Trilinos_ENABLE_Tpetra=ON \
-D Tpetra_INST_SERIAL=ON \
-D Tpetra_INST_OPENMP=OFF \
\
-D Trilinos_ENABLE_Kokkos=ON \
-D Trilinos_ENABLE_KokkosCore=ON \
-D Kokkos_ENABLE_Serial=ON \
-D Kokkos_ENABLE_OpenMP=OFF \
-D Kokkos_ENABLE_Pthread=OFF \
-D Kokkos_ENABLE_Cuda=ON \
-D Kokkos_ENABLE_Cuda_UVM=ON \
-D Kokkos_ENABLE_Cuda_Lambda=ON \
-D Kokkos_ENABLE_Cuda_Relocatable_Device_Code=OFF \
-D TPL_ENABLE_CUDA=ON \
\
-D TPL_ENABLE_MPI=ON \
-D MPI_USE_COMPILER_WRAPPERS=ON \
\
-D Tpetra_INST_CUDA:BOOL=ON \
../Trilinos
Any advice how to get past this point would be most appreciated.