GitLab

Created by: mhoemmen

@trilinos/belos @amklinv

The Tpetra specialization of Belos::MultiVecTraits::MvTransMv uses Teuchos::REDUCE_SUM. This is the only MultiVecTraits method implementation for Tpetra that uses this enum. Either PETSc or Hypre appears to define a REDUCE_SUM macro, which causes build errors when colliding with this enum. While this is really an issue with PETSc or Hypre not namespacing their macros, we should be able to deal with this in Belos, and simplify the code at the same time.

1. Keep the branch of the code for the single vector case (Tpetra::MultiVector::multiply might not optimize for that case)
2. Otherwise, call Tpetra::MultiVector::multiply. View the input SerialDenseMatrix as a MultiVector and use a replicated Map.

Here is how you create the MultiVector to view the SerialDenseMatrix.

1. Create an empty Kokkos::DualView. Get device_type from Tpetra::MultiVector.
2. Create an HostSpace Kokkos::View (FIXME: Figure out how to specify at run time that the View is unmanaged, comparable to Teuchos::rcpFromRef -- I think you can just invoke the constructor that takes a raw pointer and dimensions, and everything will be ok).
3. Set the DualView's h_view field to the new View that you created in Step 2.
4. Mark the DualView as modified on host (actually you don't have to, because of UVM).
5. Give the DualView to Tpetra::MultiVector's constructor, along with a globally replicated Map.

In the source code of Tpetra::MultiVector::multiply, this is Case 2 (where C is "local," i.e., globally replicated over the same communicator as the input MultiVectors). Belos does something weird: namely, it calls multiply on a MultiVector with a "SerialComm." Thus, the multiply() method doesn't actually get to do the communication that it knows how to do (it calls "reduce()" internally). Just let MultiVector::multiply do what it knows how to do, by letting C have a globally replicated Map with the same communicator as A and B.