Rol build errors and warnings under Cuda
Created by: crtrott
Rol doesn't build correctly when using Cuda. Basically you are not allowed to declare private (or protected) classes as members of classes which you then use as functors. In this case unaryFunc must be declared as a public member in:
/home/crtrott/Trilinos/packages/rol/adapters/tpetra/src/vector/ROL_TpetraMultiVector.hpp:193
/home/crtrott/Trilinos/packages/kokkos/core/src/Cuda/Kokkos_CudaExec.hpp(267): error: A type that is defined inside a class and has private or protected access ("ROL::TpetraMultiVector<RealT, Tpetra::Details::DefaultTypes::local_ordinal_type, Tpetra::Details::DefaultTypes::global_ordinal_type, Tpetra::Details::DefaultTypes::node_type>::unaryFunc") cannot be used in the template argument type of a __global__ function template instantiation, unless the class is local to a __device__ or __global__ function
detected during:
instantiation of "Kokkos::Impl::cuda_parallel_launch_local_memory" based on template argument <Kokkos::Impl::ParallelFor<ROL::TpetraMultiVector<RealT, Tpetra::Details::DefaultTypes::local_ordinal_type, Tpetra::Details::DefaultTypes::global_ordinal_type, Tpetra::Details::DefaultTypes::node_type>::unaryFunc, Kokkos::RangePolicy<Kokkos::CudaUVMSpace::execution_space>, Kokkos::Cuda::execution_space>>
(267): here
instantiation of "Kokkos::Impl::CudaParallelLaunch<DriverType, false>::CudaParallelLaunch(const DriverType &, const dim3 &, const dim3 &, int, cudaStream_t) [with DriverType=Kokkos::Impl::ParallelFor<ROL::TpetraMultiVector<RealT, Tpetra::Details::DefaultTypes::local_ordinal_type, Tpetra::Details::DefaultTypes::global_ordinal_type, Tpetra::Details::DefaultTypes::node_type>::unaryFunc, Kokkos::RangePolicy<Kokkos::CudaUVMSpace::execution_space>, Kokkos::Cuda::execution_space>]"
/home/crtrott/Trilinos/packages/kokkos/core/src/Cuda/Kokkos_Cuda_Parallel.hpp(541): here
instantiation of "void Kokkos::Impl::ParallelFor<FunctorType, Kokkos::RangePolicy<Traits...>, Kokkos::Cuda>::execute() const [with FunctorType=ROL::TpetraMultiVector<RealT, Tpetra::Details::DefaultTypes::local_ordinal_type, Tpetra::Details::DefaultTypes::global_ordinal_type, Tpetra::Details::DefaultTypes::node_type>::unaryFunc, Traits=<Kokkos::CudaUVMSpace::execution_space>]"
/home/crtrott/Trilinos/packages/kokkos/core/src/Kokkos_Parallel.hpp(222): here
instantiation of "void Kokkos::parallel_for(size_t, const FunctorType &, const std::string &) [with FunctorType=ROL::TpetraMultiVector<RealT, Tpetra::Details::DefaultTypes::local_ordinal_type, Tpetra::Details::DefaultTypes::global_ordinal_type, Tpetra::Details::DefaultTypes::node_type>::unaryFunc]"
/home/crtrott/Trilinos/packages/rol/adapters/tpetra/src/vector/ROL_TpetraMultiVector.hpp(266): here
instantiation of "void ROL::TpetraMultiVector<Real, LO, GO, Node>::applyUnary(const ROL::Elementwise::UnaryFunction<Real> &) [with Real=RealT, LO=Tpetra::Details::DefaultTypes::local_ordinal_type, GO=Tpetra::Details::DefaultTypes::global_ordinal_type, Node=Tpetra::Details::DefaultTypes::node_type]"
/home/crtrott/Trilinos/packages/rol/adapters/tpetra/src/vector/ROL_TpetraMultiVector.hpp(76): here
instantiation of "ROL::TpetraMultiVector<Real, LO, GO, Node>::~TpetraMultiVector() [with Real=RealT, LO=Tpetra::Details::DefaultTypes::local_ordinal_type, GO=Tpetra::Details::DefaultTypes::global_ordinal_type, Node=Tpetra::Details::DefaultTypes::node_type]"
/home/crtrott/Trilinos/packages/rol/adapters/tpetra/test/vector/test_01.cpp(108): here
There are more of those kind of errors.
Then there are also warnings:
/home/crtrott/Trilinos/packages/rol/adapters/tpetra/src/vector/ROL_TpetraMultiVector.hpp(205): warning: calling a __host__ function("apply") from a __host__ __device__ function("operator()") is not allowed
Essentially it looks like there are some KOKKOS_INLINE_FUNCTION markings missing.