Fix 1571 (CMake variable & detection of MPI CUDA awareness)
Created by: mhoemmen
@trilinos/tpetra Hi all! I wrote a PR that does the following:
- Defines a CMake variable,
Tpetra_ASSUME_CUDA_AWARE_MPI
, for whether Tpetra may assume that the MPI implementation is CUDA aware. - Attempts to detect the default value of this variable (this currently only works with OpenMPI; it currently safely assumes OFF for other MPI implementations).
- Enforces
CUDA_VERSION
>= 7.5.
I took particular care to avoid breaking the cross-compilation case. The PR explicitly checks CMAKE_CROSSCOMPILING
; if ON
, Tpetra does not attempt to run executables. This is relevant because I know users who do cross-compilation with CUDA builds right now.
I welcome feedback! My only concern is that the PR forces Tpetra to decide whether MPI is CUDA aware at configure time. Some MPI implementations, like MVAPICH (see http://mvapich.cse.ohio-state.edu/userguide/gdr/2.2/ ), let users control this at run time, by setting an environment variable. This means that Tpetra may also need run-time environment variable control. However, we can always add that feature later. The CMake option is still useful, because it could determine the environment variable's default value. Thus, I think the PR is fine as it stands.