Skip to content

Tpetra: Fix compiling Tpetra::Multivector::dot() with CUDA for complex Scalar

James Willenbring requested to merge masterleinad:fix_tpetra_multivector_dot_complex_cuda into develop

Created by: masterleinad

@trilinos/tpetra

Description

This pull request removes Tpetra::Details::localDotWorkAround and calls KokkosBlas::dot unconditionally.

Motivation and Context

Similar to #4312, without this patch I get an error message similar to

include/Cuda/Kokkos_Cuda_Parallel.hpp(2461): error: calling a __host__ function("") from a __device__ function("Kokkos::Impl::ParallelReduce< ::Kokkos::Impl::CudaFunctorAdapter< ::,  ::Kokkos::RangePolicy< ::Kokkos::Cuda, int > ,         double, void> ,  ::Kokkos::RangePolicy< ::Kokkos::Cuda, int > ,  ::Kokkos::InvalidType,  ::Kokkos::Cuda> ::operator ()") is not allowed

caused by the call to Kokkos::parallel_reduce in Tpetra::Details::localDotWorkAround for complex Scalar type when compiling with nvcc. I tried both nvcc-8 and nvcc-10.

The workaround was first introduced in #4253 to allow compiling with std::complex scalar. The comments mention that the workaround is likely not necessary after https://github.com/kokkos/kokkos-kernels/pull/374 and https://github.com/kokkos/kokkos-kernels/pull/374 are merged. This was done on January 23 and January 24 respectively and the follow-up pull request https://github.com/kokkos/kokkos-kernels/pull/380 was merged on January 29.

Related Issues

How Has This Been Tested?

Compiling deal.II with CUDA and Tpetra support in Trilinos for std::complex scalar type works and all related deal.II tests are passing.

Checklist

  • My commit messages mention the appropriate GitHub issue numbers.
  • My code follows the code style of the affected package(s).
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the code contribution guidelines for this project.
  • I have added tests to cover my changes.
  • All new and existing tests passed.
  • No new compiler warnings were introduced.
  • These changes break backwards compatibility.

Merge request reports