Tpetra,Stokhos: Fix #4870 (prefer running MV norms on device)
Created by: mhoemmen
@trilinos/tpetra @trilinos/stokhos @vbrunini @etphipp
Supersedes PR #4881, PR #4871, and PR #4885.
Description
- Make
Tpetra::MultiVector
prefer computing the local (to each MPI process) parts of norms (norm1
,norm2
,normInf
) on device. Do so even if the MultiVector is currently sync'd to host, unless the MultiVector has too few local rows. - Factor out the implementation of
Tpetra::MultiVector::norm*
into a separate function,Tpetra::Details::normImpl
. Do ETI for that function, when ETI is enabled. - Make Stokhos do ETI for
Tpetra::Details::normImpl
, for itsUQ::PCE
andMP::Vector
Scalar types.
To elaborate the last point: Tpetra's ETI macro for Tpetra::Details::normImpl
only takes 2 template parameters: Scalar
and Node
. This meant that I couldn't use Stokhos' ETI system directly (see TPETRA_ETI_CLASSES
in stokhos/src/CMakeLists.txt
). I had to take different approaches for PCE types vs. MP::Vector
types. My solution for PCE types looks more permanent (read "is less of a hack") than my solution for MP::Vector
types; it would be great if @etphipp could comment.
Motivation and Context
- SPARC performance
- Build time
Related Issues
- Closes #4870 (closed)