Skip to content

GitLab

  • Menu
Projects Groups Snippets
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • T Trilinos
  • Project information
    • Project information
    • Activity
    • Labels
    • Planning hierarchy
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 936
    • Issues 936
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 22
    • Merge requests 22
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • James Willenbring
  • Trilinos
  • Issues
  • #3469

Closed
Open
Created Sep 19, 2018 by James Willenbring@jmwilleMaintainer

Configuration of CUDA-enabled build fails

Created by: pelesh

Configuration of a CUDA-enabled Tpetra/Kokkos build fails at the point where CMake checks for C++ compiler. The error message is:

  Linking CXX executable cmTC_babeb

  /usr/tce/packages/cmake/cmake-3.5.2/bin/cmake -E cmake_link_script
  CMakeFiles/cmTC_babeb.dir/link.txt --verbose=1

  /usr/tce/packages/openmpi/openmpi-2.0.0-gcc-6.1.0/bin/mpicxx
  CMakeFiles/cmTC_babeb.dir/testCXXCompiler.cxx.o -o cmTC_babeb -rdynamic

  nvcc fatal : Unknown option 'rdynamic,-fexceptions,-pthread'

  gmake[1]: *** [cmTC_babeb] Error 1

It seems as if Kokkos' nvc_wrapper does not insert -Xlinker flags correctly.

I am building from current master branch on a system with following configuration:

$ module list

Currently Loaded Modules:
1) StdEnv   2) gcc/6.1.0   3) cuda/9.1.85   4) openmpi/2.0.0   5) cmake/3.5.2

The same issue occurs with different openmpi, CUDA, CMake and gcc versions.

I am following Tpetra build instructions from $Trilinos/packages/tpetra/doc/FAQ.txt, and I set environment variables like this:

  export OMPI_CXX=${Trilinos}/packages/kokkos/bin/nvcc_wrapper
  export CUDA_LAUNCH_BLOCKING=1
  export CUDA_MANAGED_FORCE_DEVICE_ALLOC=1

I use following configuration script to set cmake options:

#!/bin/bash

cmake \
   -D CMAKE_C_COMPILER="mpicc" \
   -D CMAKE_CXX_COMPILER="mpicxx" \
   -D CMAKE_Fortran_COMPILER="mpif90" \
   \
   -D Trilinos_CXX11_FLAGS="-std=c++11 --expt-extended-lambda" \
   \
   -D Trilinos_ENABLE_OpenMP=OFF \
   -D TPL_ENABLE_Pthread=OFF \
   \
   -D Trilinos_ENABLE_Teuchos=ON \
   -D Trilinos_ENABLE_Tpetra=ON \
   -D Tpetra_INST_SERIAL=ON \
   -D Tpetra_INST_OPENMP=OFF \
   \
   -D Trilinos_ENABLE_Kokkos=ON \
   -D Trilinos_ENABLE_KokkosCore=ON \
   -D Kokkos_ENABLE_Serial=ON \
   -D Kokkos_ENABLE_OpenMP=OFF \
   -D Kokkos_ENABLE_Pthread=OFF \
   -D Kokkos_ENABLE_Cuda=ON \
   -D Kokkos_ENABLE_Cuda_UVM=ON \
   -D Kokkos_ENABLE_Cuda_Lambda=ON \
   -D Kokkos_ENABLE_Cuda_Relocatable_Device_Code=OFF \
   -D TPL_ENABLE_CUDA=ON \
   \
   -D TPL_ENABLE_MPI=ON \
   -D MPI_USE_COMPILER_WRAPPERS=ON \
   \
  -D Tpetra_INST_CUDA:BOOL=ON \
../Trilinos

Any advice how to get past this point would be most appreciated.

Assignee
Assign to
Time tracking