Skip to content

Fix 1571 (CMake variable & detection of MPI CUDA awareness)

James Willenbring requested to merge Fix-1571 into develop

Created by: mhoemmen

@trilinos/tpetra Hi all! I wrote a PR that does the following:

  1. Defines a CMake variable, Tpetra_ASSUME_CUDA_AWARE_MPI, for whether Tpetra may assume that the MPI implementation is CUDA aware.
  2. Attempts to detect the default value of this variable (this currently only works with OpenMPI; it currently safely assumes OFF for other MPI implementations).
  3. Enforces CUDA_VERSION >= 7.5.

I took particular care to avoid breaking the cross-compilation case. The PR explicitly checks CMAKE_CROSSCOMPILING; if ON, Tpetra does not attempt to run executables. This is relevant because I know users who do cross-compilation with CUDA builds right now.

I welcome feedback! My only concern is that the PR forces Tpetra to decide whether MPI is CUDA aware at configure time. Some MPI implementations, like MVAPICH (see http://mvapich.cse.ohio-state.edu/userguide/gdr/2.2/ ), let users control this at run time, by setting an environment variable. This means that Tpetra may also need run-time environment variable control. However, we can always add that feature later. The CMake option is still useful, because it could determine the environment variable's default value. Thus, I think the PR is fine as it stands.

Merge request reports