Tpetra compile error build with serial build + complex types with gcc/7.2 and gcc/7.3
Created by: ndellingwood
This error reported by @hkthorn
Tpetra fails to compile with complex types with gcc/7.2
and gcc/7.3
and Tpetra_INST_SERIAL:BOOL=ON
; my configuration is posted at end of this issue. Error occurs in Tpetra's AbsMaxOp
functor's operator () at line 755 when calling Kokkos::Impl::atomic_fetch_oper
.
Not sure if this is a Kokkos issue or something issue with execution space defaults.
I explicitly enabled only the Tpetra Serial backend and no Kokkos configure options were used on the cmake line, but the compilation error shows Kokkos::OpenMP
as the execution space expected by AbsMaxOp
which seems related to the error. Shouldn't the execution space in Tpetra and Kokkos agree if I only set Tpetra_INST_SERIAL:BOOL=ON
and no Kokkos options?
Looking at code surrounding the error line 755 in this code block, the explicit specialization of AbsMaxOp
within the KOKKOS_ENABLE_SERIAL
guards is not called. Going back to previous question about default Kokkos execution space settings when Tpetra's execution space is explicitly set, I would expect that KOKKOS_ENABLE_SERIAL=ON
be true if I only set Tpetra_INST_SERIAL:BOOL=ON
, though this may be unrelated.
@hkthorn mentioned this configuration worked for her in the past. A Gblame on the error line shows it was modified by PR #3999, adding @mhoemmen and @rppawlo.
Edit: Strike out irrelevant info due to careless copy/paste of configuration options I made.
@trilinos/tpetra @trilinos/kokkos
Error message:
/home/ndellin/trilinos-ssh/Trilinos/packages/tpetra/core/src/kokkos_refactor/Tpetra_KokkosRefactor_Details_MultiVectorDistObjectKernels.hpp:755:39: error: no matching function for call to ‘atomic_fetch_oper(Tpetra::KokkosRefactor::Details::AbsMaxOper<Kokkos::complex<double>, Kokkos::complex<double> >, Kokkos::complex<double>*, const Kokkos::complex<double>&)’
Kokkos::Impl::atomic_fetch_oper (AbsMaxOper<Scalar,Scalar>(), &dest, src);
In file included from /home/ndellin/trilinos-ssh/Trilinos/packages/kokkos/core/src/Kokkos_Atomic.hpp:288:0,
from /home/ndellin/trilinos-ssh/Trilinos/packages/kokkos/core/src/impl/Kokkos_Atomic_View.hpp:47,
from /home/ndellin/trilinos-ssh/Trilinos/packages/kokkos/core/src/impl/Kokkos_ViewMapping.hpp:56,
from /home/ndellin/trilinos-ssh/Trilinos/packages/kokkos/core/src/Kokkos_View.hpp:451,
from /home/ndellin/trilinos-ssh/Trilinos/packages/kokkos/core/src/Kokkos_Parallel.hpp:52,
from /home/ndellin/trilinos-ssh/Trilinos/packages/kokkos/core/src/Kokkos_Serial.hpp:55,
from /home/ndellin/trilinos-ssh/Trilinos/packages/kokkos/core/src/Kokkos_Core.hpp:53,
from /home/ndellin/trilinos-ssh/Trilinos/packages/teuchos/kokkoscompat/src/KokkosCompat_ClassicNodeAPI_Wrapper.hpp:5,
from /home/ndellin/trilinos-ssh/Trilinos/packages/tpetra/core/src/Tpetra_Details_DefaultTypes.hpp:47,
from /home/ndellin/trilinos-ssh/Trilinos/packages/tpetra/core/src/Tpetra_ConfigDefs.hpp:45,
from /home/ndellin/trilinos-ssh/Trilinos/packages/tpetra/core/src/Tpetra_Core.hpp:54,
from /home/ndellin/trilinos-ssh/Trilinos/packages/tpetra/core/test/Tpetra_TestingUtilities.hpp:55,
from /home/ndellin/trilinos-ssh/Trilinos/packages/tpetra/core/test/Block/BlockMultiVector.cpp:44:
/home/ndellin/trilinos-ssh/Trilinos/packages/kokkos/core/src/impl/Kokkos_Atomic_Generic.hpp:155:3: note: candidate: template<class Oper, class T> T Kokkos::Impl::atomic_fetch_oper(const Oper&, volatile T*, typename Kokkos::Impl::enable_if<((sizeof (T) != sizeof (int)) && (sizeof (T) == sizeof (long long unsigned int))), const T>::type)
T atomic_fetch_oper( const Oper& op, volatile T * const dest ,
^~~~~~~~~~~~~~~~~
/home/ndellin/trilinos-ssh/Trilinos/packages/kokkos/core/src/impl/Kokkos_Atomic_Generic.hpp:155:3: note: template argument deduction/substitution failed:
/home/ndellin/trilinos-ssh/Trilinos/packages/kokkos/core/src/impl/Kokkos_Atomic_Generic.hpp: In substitution of ‘template<class Oper, class T> T Kokkos::Impl::atomic_fetch_oper(const Oper&, volatile T*, typename Kokkos::Impl::enable_if<((sizeof (T) != sizeof (int)) && (sizeof (T) == sizeof (long long unsigned int))), const T>::type) [with Oper = Tpetra::KokkosRefactor::Details::AbsMaxOper<Kokkos::complex<double>, Kokkos::complex<double> >; T = Kokkos::complex<double>]’:
/home/ndellin/trilinos-ssh/Trilinos/packages/tpetra/core/src/kokkos_refactor/Tpetra_KokkosRefactor_Details_MultiVectorDistObjectKernels.hpp:755:39: required from ‘void Tpetra::KokkosRefactor::Details::AbsMaxOp<ExecutionSpace>::operator()(Scalar&, const Scalar&) const [with Scalar = Kokkos::complex<double>; ExecutionSpace = Kokkos::OpenMP]’
/home/ndellin/trilinos-ssh/Trilinos/packages/tpetra/core/src/kokkos_refactor/Tpetra_KokkosRefactor_Details_MultiVectorDistObjectKernels.hpp:915:11: required from ‘void Tpetra::KokkosRefactor::Details::UnpackArrayMultiColumnWithBoundsCheck<ExecutionSpace, DstView, SrcView, IdxView, Op, SizeType>::operator()(const size_type&, Tpetra::KokkosRefactor::Details::UnpackArrayMultiColumnWithBoundsCheck<ExecutionSpace, DstView, SrcView, IdxView, Op, SizeType>::value_type&) const [with ExecutionSpace = Kokkos::OpenMP; DstView = Kokkos::View<Kokkos::complex<double>**, Kokkos::LayoutLeft, Kokkos::OpenMP>; SrcView = Kokkos::View<Kokkos::complex<double>*, Kokkos::LayoutRight, Kokkos::Device<Kokkos::OpenMP, Kokkos::HostSpace> >; IdxView = Kokkos::View<int*, Kokkos::LayoutRight, Kokkos::Device<Kokkos::OpenMP, Kokkos::HostSpace> >; Op = Tpetra::KokkosRefactor::Details::AbsMaxOp<Kokkos::OpenMP>; SizeType = long unsigned int; Tpetra::KokkosRefactor::Details::UnpackArrayMultiColumnWithBoundsCheck<ExecutionSpace, DstView, SrcView, IdxView, Op, SizeType>::size_type = long unsigned int; Tpetra::KokkosRefactor::Details::UnpackArrayMultiColumnWithBoundsCheck<ExecutionSpace, DstView, SrcView, IdxView, Op, SizeType>::value_type = int]’
/home/ndellin/trilinos-ssh/Trilinos/packages/kokkos/core/src/OpenMP/Kokkos_OpenMP_Parallel.hpp:327:16: required from ‘static typename std::enable_if<std::is_same<_Dummy, void>::value>::type Kokkos::Impl::ParallelReduce<FunctorType, Kokkos::RangePolicy<Traits ...>, ReducerType, Kokkos::OpenMP>::exec_range(const FunctorType&, Kokkos::Impl::ParallelReduce<FunctorType, Kokkos::RangePolicy<Traits ...>, ReducerType, Kokkos::OpenMP>::Member, Kokkos::Impl::ParallelReduce<FunctorType, Kokkos::RangePolicy<Traits ...>, ReducerType, Kokkos::OpenMP>::Member, Kokkos::Impl::ParallelReduce<FunctorType, Kokkos::RangePolicy<Traits ...>, ReducerType, Kokkos::OpenMP>::reference_type) [with TagType = void; FunctorType = Tpetra::KokkosRefactor::Details::UnpackArrayMultiColumnWithBoundsCheck<Kokkos::OpenMP, Kokkos::View<Kokkos::complex<double>**, Kokkos::LayoutLeft, Kokkos::OpenMP>, Kokkos::View<Kokkos::complex<double>*, Kokkos::LayoutRight, Kokkos::Device<Kokkos::OpenMP, Kokkos::HostSpace> >, Kokkos::View<int*, Kokkos::LayoutRight, Kokkos::Device<Kokkos::OpenMP, Kokkos::HostSpace> >, Tpetra::KokkosRefactor::Details::AbsMaxOp<Kokkos::OpenMP>, long unsigned int>; ReducerType = Kokkos::InvalidType; Traits = {Kokkos::OpenMP, long unsigned int}; typename std::enable_if<std::is_same<_Dummy, void>::value>::type = void; Kokkos::Impl::ParallelReduce<FunctorType, Kokkos::RangePolicy<Traits ...>, ReducerType, Kokkos::OpenMP>::Member = long unsigned int; Kokkos::Impl::ParallelReduce<FunctorType, Kokkos::RangePolicy<Traits ...>, ReducerType, Kokkos::OpenMP>::reference_type = int&]’
/home/ndellin/trilinos-ssh/Trilinos/packages/kokkos/core/src/OpenMP/Kokkos_OpenMP_Parallel.hpp:391:34: required from ‘void Kokkos::Impl::ParallelReduce<FunctorType, Kokkos::RangePolicy<Traits ...>, ReducerType, Kokkos::OpenMP>::execute() const [with FunctorType = Tpetra::KokkosRefactor::Details::UnpackArrayMultiColumnWithBoundsCheck<Kokkos::OpenMP, Kokkos::View<Kokkos::complex<double>**, Kokkos::LayoutLeft, Kokkos::OpenMP>, Kokkos::View<Kokkos::complex<double>*, Kokkos::LayoutRight, Kokkos::Device<Kokkos::OpenMP, Kokkos::HostSpace> >, Kokkos::View<int*, Kokkos::LayoutRight, Kokkos::Device<Kokkos::OpenMP, Kokkos::HostSpace> >, Tpetra::KokkosRefactor::Details::AbsMaxOp<Kokkos::OpenMP>, long unsigned int>; ReducerType = Kokkos::InvalidType; Traits = {Kokkos::OpenMP, long unsigned int}]’
Configuration:
module load sems-gcc/7.2.0 sems-cmake/3.12.2
rm -rf CMake*
cmake \
-DCMAKE_C_COMPILER=`which gcc` \
-DCMAKE_CXX_COMPILER=`which g++` \
-DCMAKE_Fortran_COMPILER=`which gfortran` \
-DCMAKE_CXX_FLAGS="-Wall -O3 -DNDEBUG" \
-DCMAKE_C_FLAGS="-Wall -O3 -DNDEBUG" \
-D Trilinos_ENABLE_Belos:BOOL=ON \
-D Trilinos_ENABLE_Anasazi:BOOL=ON \
-D Trilinos_ENABLE_AztecOO:BOOL=ON \
-D Trilinos_ENABLE_Teuchos:BOOL=ON \
-D Trilinos_ENABLE_Ifpack:BOOL=ON \
-D Trilinos_ENABLE_Epetra:BOOL=ON \
-D Trilinos_ENABLE_EpetraExt:BOOL=ON \
-D Trilinos_ENABLE_Triutils:BOOL=ON \
-D Trilinos_ENABLE_Tpetra:BOOL=ON \
-D Tpetra_INST_SERIAL:BOOL=ON \
-D Trilinos_ENABLE_ALL_OPTIONAL_PACKAGES:BOOL=OFF \
-D Teuchos_ENABLE_COMPLEX:BOOL=ON \
-D Trilinos_ENABLE_COMPLEX:BOOL=ON \
-D Trilinos_ENABLE_OpenMP:BOOL=ON \
-D CMAKE_BUILD_TYPE:STRING=DEBUG \
-D CMAKE_VERBOSE_MAKEFILE:BOOL=ON \
-D Trilinos_ENABLE_EXAMPLES:BOOL=ON \
-D Trilinos_ENABLE_TESTS:BOOL=ON \
-DTPL_ENABLE_BLAS=ON \
-DTPL_ENABLE_LAPACK=ON \
../../