Trilinos issueshttps://gitlab.osti.gov/jmwille/Trilinos/-/issues2016-03-10T12:44:55Zhttps://gitlab.osti.gov/jmwille/Trilinos/-/issues/190Tpetra: Configure passes with no Node enabled2016-03-10T12:44:55ZJames WillenbringTpetra: Configure passes with no Node enabled*Created by: mhoemmen*
Moved from Bugzilla Bug 6474:
https://software.sandia.gov/bugzilla/show_bug.cgi?id=6474
@aprokop reported on 20 Nov 2015:
If ETI is on, and Tpetra_INST_Serial=OFF, Tpetra reports that all nodes are disabled, ...*Created by: mhoemmen*
Moved from Bugzilla Bug 6474:
https://software.sandia.gov/bugzilla/show_bug.cgi?id=6474
@aprokop reported on 20 Nov 2015:
If ETI is on, and Tpetra_INST_Serial=OFF, Tpetra reports that all nodes are disabled, but does not fail to configure. It really should! Configure script:
-D Trilinos_ENABLE_OpenMP=OFF
-D Tpetra_INST_SERIAL=OFF
-D Trilinos_ENABLE_EXPLICIT_INSTANTIATION=ON
-D Tpetra_INST_INT_INT=OFF
-D Tpetra_INST_INT_LONG=OFF
-D Tpetra_INST_INT_LONG_LONG=ON
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/202STKUnit_tests_util_parallel_UnitTest_MPI_4 failing almost everywhere (includi...2016-03-10T16:07:47ZJames WillenbringSTKUnit_tests_util_parallel_UnitTest_MPI_4 failing almost everywhere (including ATTB machine)*Created by: bartlettroscoe*
The test STKUnit_tests_util_parallel_UnitTest_MPI_4 is failing on all but one build on one automated test platform as shown in [this CDash query](http://testing.sandia.gov/cdash/queryTests.php?project=Trilin...*Created by: bartlettroscoe*
The test STKUnit_tests_util_parallel_UnitTest_MPI_4 is failing on all but one build on one automated test platform as shown in [this CDash query](http://testing.sandia.gov/cdash/queryTests.php?project=Trilinos&date=&limit=200&filtercount=3&showfilters=1&limit=200&filtercombine=and&field1=testname/string&compare1=65&value1=STKUnit_tests_util_parallel_UnitTest&field2=buildstarttime/date&compare2=83&value2=2016-03-08&field3=buildstarttime/date&compare3=84&value3=2016-03-11). I am also here to report that it also fails on the Advanced Technology Test Best machine hansen (which is an important platform for obvious reasons).
In fact, this test has been failing for as far back as we have data (currently 2016-02-05) as shown in [this query](http://testing.sandia.gov/cdash/queryTests.php?project=Trilinos&date=&limit=200&filtercount=3&showfilters=1&limit=200&filtercombine=and&field1=testname/string&compare1=65&value1=STKUnit_tests_util_parallel_UnitTest&field2=buildstarttime/date&compare2=83&value2=2016-01-01&field3=buildstarttime/date&compare3=84&value3=2016-03-11).
Who is responsible for STK under Trilinos?
CC: @trilinos/framework
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/198Kokkos Serial execution space is not tested with TeamPolicy.2016-03-10T17:45:34ZJames WillenbringKokkos Serial execution space is not tested with TeamPolicy.*Created by: kyungjoo-kim*
The following error comes from when I use Serial space (OpenMP space is okay). I think that the TeamPolicy does not work here. The Tpetra block crs matrix is already submitted to Christian's Trilinos repo and ...*Created by: kyungjoo-kim*
The following error comes from when I use Serial space (OpenMP space is okay). I think that the TeamPolicy does not work here. The Tpetra block crs matrix is already submitted to Christian's Trilinos repo and you can reproduce this error in CI testing.
/home/kyukim/Work/lib/trilinos/christian/packages/kokkos/core/src/Kokkos_Serial.hpp:672:9: error: no match for call to ‘(const Tpetra::Experimental::BlockCrsMatrix<S, LO, GO, N>::localApplyBlockNoTrans(const Tpetra::Experimental::BlockMultiVector<Scalar, LO, GO, Node>&, Tpetra::Experimental::BlockMultiVector<Scalar, LO, GO, Node>&, Scalar, Scalar) [with Scalar = double; LO = int; GO = int; Node = Kokkos::Compat::KokkosDeviceWrapperNodeKokkos::Serial]::__lambda2) (Kokkos::Impl::ParallelFor<Tpetra::Experimental::BlockCrsMatrix<S, LO, GO, N>::localApplyBlockNoTrans(const Tpetra::Experimental::BlockMultiVector<Scalar, LO, GO, Node>&, Tpetra::Experimental::BlockMultiVector<Scalar, LO, GO, Node>&, Scalar, Scalar) [with Scalar = double; LO = int; GO = int; Node = Kokkos::Compat::KokkosDeviceWrapperNodeKokkos::Serial]::__lambda2, Kokkos::TeamPolicyKokkos::Schedule<Kokkos::Dynamic, Kokkos::Serial>, Kokkos::Serial>::Member)’
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/135Failed build with trilinos-release-12-62016-03-11T00:31:50ZJames WillenbringFailed build with trilinos-release-12-6*Created by: BarrySmith*
I'm unable to build the trilinos-release-12-6-branch
Obtaining errors of the form
../../../../../tpetra/core/src/libtpetra.so.12.6: undefined reference to `KokkosBlas::Impl::Update<Kokkos::View<int con\
st**...*Created by: BarrySmith*
I'm unable to build the trilinos-release-12-6-branch
Obtaining errors of the form
../../../../../tpetra/core/src/libtpetra.so.12.6: undefined reference to `KokkosBlas::Impl::Update<Kokkos::View<int con\
st**, Kokkos::LayoutLeft, Kokkos::Device<Kokkos::Serial, Kokkos::HostSpace>, Kokkos::MemoryTraits<1u>, Kokkos::Impl::Vi\
ewDefault>, Kokkos::View<int const**, Kokkos::LayoutLeft, Kokkos::Device<Kokkos::Serial, Kokkos::HostSpace>, Kokkos::Me\
moryTraits<1u>, Kokkos::Impl::ViewDefault>, Kokkos::View<int**, Kokkos::LayoutLeft, Kokkos::Device<Kokkos::Serial, Kokk\
os::HostSpace>, Kokkos::MemoryTraits<1u>, Kokkos::Impl::ViewDefault>, 2>::update(int const&, Kokkos::View<int const**, \
Kokkos::LayoutLeft, Kokkos::Device<Kokkos::Serial, Kokkos::HostSpace>, Kokkos::MemoryTraits<1u>, Kokkos::Impl::ViewDefa\
ult> const&, int const&, Kokkos::View<int const**, Kokkos::LayoutLeft, Kokkos::Device<Kokkos::Serial, Kokkos::HostSpace\
> , Kokkos::MemoryTraits<1u>, Kokkos::Impl::ViewDefault> const&, int const&, Kokkos::View<int**, Kokkos::LayoutLeft, Kok\
> kos::Device<Kokkos::Serial, Kokkos::HostSpace>, Kokkos::MemoryTraits<1u>, Kokkos::Impl::ViewDefault> const&)'
> ../../../../../tpetra/core/src/libtpetra.so.12.6: undefined reference to `KokkosSparse::Impl::SPMV_MV<double const*, Ko\
> kkos::LayoutLeft, Kokkos::Device<Kokkos::Serial, Kokkos::HostSpace>, Kokkos::MemoryTraits<1u>, double const, int, Kokko\
> s::Device<Kokkos::Serial, Kokkos::HostSpace>, Kokkos::MemoryTraits<1u>, unsigned long, double const**, Kokkos::LayoutLe\
> ft, Kokkos::Device<Kokkos::Serial, Kokkos::HostSpace>, Kokkos::MemoryTraits<3u>, double const*, Kokkos::LayoutLeft, Kok\
> kos::Device<Kokkos::Serial, Kokkos::HostSpace>, Kokkos::MemoryTraits<1u>, double**, Kokkos::LayoutLeft, Kokkos::Device<\
> Kokkos::Serial, Kokkos::HostSpace>, Kokkos::MemoryTraits<1u> >::spmv_mv(char const_, double const&, KokkosSparse::CrsMa\
> trix<double const, int, Kokkos::Device<Kokkos::Serial, Kokkos::HostSpace>, Kokkos::MemoryTraits<1u>, unsigned long> con\
> st&, Kokkos::View<double const__, Kokkos::LayoutLeft, Kokkos::Device<Kokkos::Serial, Kokkos::HostSpace>, Kokkos::Memory\
> Traits<3u>, Kokkos::Impl::ViewDefault> const&, double const&, Kokkos::View<double_*, Kokkos::LayoutLeft, Kokkos::Device\
> <Kokkos::Serial, Kokkos::HostSpace>, Kokkos::MemoryTraits<1u>, Kokkos::Impl::ViewDefault> const&)'
Full log including configure options and machine configuration can be found at
ftp://ftp.mcs.anl.gov/pub/petsc/trilinos-release-12-6-failure
The same configure options have worked successfully with Trilinos 'master' branch.
@jwillenbring
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/204Missing instantiations in muelu-adapters2016-03-11T16:54:17ZJames WillenbringMissing instantiations in muelu-adapters*Created by: tawiesn*
If, e.g., Tpetra is instantiated both on GO=int and GO=long long, the <double, int, int, ...> instantiations of classes in muelu-adapters (such as Xpetra::LinearOpBase, and MueLuPreconditionerFactory)
*Created by: tawiesn*
If, e.g., Tpetra is instantiated both on GO=int and GO=long long, the <double, int, int, ...> instantiations of classes in muelu-adapters (such as Xpetra::LinearOpBase, and MueLuPreconditionerFactory)
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/209IFPACK2 Compile Failure - GCC 4.8.4 on Hansen Test System2016-03-11T19:39:11ZJames WillenbringIFPACK2 Compile Failure - GCC 4.8.4 on Hansen Test System*Created by: nmhamster*
Reproducer:
`module load devpack/openmpi/1.10.0/gcc/4.8.4/cuda/7.5.18` (Hansen)
Configure Trilinos, Run `make` and output:
```
[ 90%] Building CXX object packages/ifpack2/src/CMakeFiles/ifpack2.dir/Ifpack2_Add...*Created by: nmhamster*
Reproducer:
`module load devpack/openmpi/1.10.0/gcc/4.8.4/cuda/7.5.18` (Hansen)
Configure Trilinos, Run `make` and output:
```
[ 90%] Building CXX object packages/ifpack2/src/CMakeFiles/ifpack2.dir/Ifpack2_AdditiveSchwarz_Serial.cpp.o
In file included from /home/sdhammo/git/trilinos-github-repo/build-devpack-gcc-4.8.4/packages/ifpack2/src/Ifpack2_AdditiveSchwarz_Serial.cpp:50:0:
/home/sdhammo/git/trilinos-github-repo/packages/ifpack2/src/Ifpack2_AdditiveSchwarz_def.hpp: In instantiation of ‘void Ifpack2::AdditiveSchwarz<MatrixType, LocalInverseType>::setup() [with MatrixType = Tpetra::RowMatrix<double, int, long int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial> >; LocalInverseType = Ifpack2::Preconditioner<double, int, long int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial> >]’:
/home/sdhammo/git/trilinos-github-repo/build-devpack-gcc-4.8.4/packages/ifpack2/src/Ifpack2_AdditiveSchwarz_Serial.cpp:61:1: required from here
/home/sdhammo/git/trilinos-github-repo/packages/ifpack2/src/Ifpack2_AdditiveSchwarz_def.hpp:1531:75: error: conversion from ‘Teuchos::ArrayRCP<long int>’ to non-scalar type ‘Teuchos::ArrayRCP<int>’ requested
ArrayRCP<local_ordinal_type> perm = sol.getPermutationRCPConst (true);
^
/home/sdhammo/git/trilinos-github-repo/packages/ifpack2/src/Ifpack2_AdditiveSchwarz_def.hpp:1532:74: error: conversion from ‘Teuchos::ArrayRCP<long int>’ to non-scalar type ‘Teuchos::ArrayRCP<int>’ requested
ArrayRCP<local_ordinal_type> revperm = sol.getPermutationRCPConst ();
^
In file included from /home/sdhammo/git/trilinos-github-repo/packages/zoltan2/src/algorithms/order/Zoltan2_OrderingAlgorithms.hpp:51:0,
from /home/sdhammo/git/trilinos-github-repo/packages/zoltan2/src/problems/Zoltan2_OrderingProblem.hpp:54,
from /home/sdhammo/git/trilinos-github-repo/packages/ifpack2/src/Ifpack2_AdditiveSchwarz_def.hpp:68,
from /home/sdhammo/git/trilinos-github-repo/build-devpack-gcc-4.8.4/packages/ifpack2/src/Ifpack2_AdditiveSchwarz_Serial.cpp:50:
/home/sdhammo/git/trilinos-github-repo/packages/zoltan2/src/algorithms/order/Zoltan2_AlgRCM.hpp: In instantiation of ‘int Zoltan2::AlgRCM<Adapter>::order(const Teuchos::RCP<Zoltan2::OrderingSolution<typename Adapter::lno_t, typename Adapter::gno_t> >&) [with Adapter = Zoltan2::MatrixAdapter<Xpetra::RowMatrix<double, int, long int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial> >, Xpetra::RowMatrix<double, int, long int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial> > >; typename Adapter::gno_t = long int; typename Adapter::lno_t = int]’:
/home/sdhammo/git/trilinos-github-repo/packages/zoltan2/src/problems/Zoltan2_OrderingProblem.hpp:199:7: required from ‘void Zoltan2::OrderingProblem<Adapter>::solve(bool) [with Adapter = Zoltan2::XpetraRowMatrixAdapter<Xpetra::RowMatrix<double, int, long int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial> >, Xpetra::RowMatrix<double, int, long int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial> > >]’
/home/sdhammo/git/trilinos-github-repo/packages/ifpack2/src/Ifpack2_AdditiveSchwarz_def.hpp:1517:5: required from ‘void Ifpack2::AdditiveSchwarz<MatrixType, LocalInverseType>::setup() [with MatrixType = Tpetra::RowMatrix<double, int, long int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial> >; LocalInverseType = Ifpack2::Preconditioner<double, int, long int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial> >]’
/home/sdhammo/git/trilinos-github-repo/build-devpack-gcc-4.8.4/packages/ifpack2/src/Ifpack2_AdditiveSchwarz_Serial.cpp:61:1: required from here
/home/sdhammo/git/trilinos-github-repo/packages/zoltan2/src/algorithms/order/Zoltan2_AlgRCM.hpp:112:69: error: conversion from ‘const Teuchos::ArrayRCP<long int>’ to non-scalar type ‘const Teuchos::ArrayRCP<int>’ requested
const ArrayRCP<lno_t> invPerm = solution->getPermutationRCP(true);
^
In file included from /home/sdhammo/git/trilinos-github-repo/packages/zoltan2/src/algorithms/order/Zoltan2_OrderingAlgorithms.hpp:49:0,
from /home/sdhammo/git/trilinos-github-repo/packages/zoltan2/src/problems/Zoltan2_OrderingProblem.hpp:54,
from /home/sdhammo/git/trilinos-github-repo/packages/ifpack2/src/Ifpack2_AdditiveSchwarz_def.hpp:68,
from /home/sdhammo/git/trilinos-github-repo/build-devpack-gcc-4.8.4/packages/ifpack2/src/Ifpack2_AdditiveSchwarz_Serial.cpp:50:
/home/sdhammo/git/trilinos-github-repo/packages/zoltan2/src/algorithms/order/Zoltan2_AlgNatural.hpp: In instantiation of ‘int Zoltan2::AlgNatural<Adapter>::order(const Teuchos::RCP<Zoltan2::OrderingSolution<typename Adapter::lno_t, typename Adapter::gno_t> >&) [with Adapter = Zoltan2::MatrixAdapter<Xpetra::RowMatrix<double, int, long int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial> >, Xpetra::RowMatrix<double, int, long int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial> > >; typename Adapter::gno_t = long int; typename Adapter::lno_t = int]’:
/home/sdhammo/git/trilinos-github-repo/packages/zoltan2/src/problems/Zoltan2_OrderingProblem.hpp:205:7: required from ‘void Zoltan2::OrderingProblem<Adapter>::solve(bool) [with Adapter = Zoltan2::XpetraRowMatrixAdapter<Xpetra::RowMatrix<double, int, long int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial> >, Xpetra::RowMatrix<double, int, long int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial> > >]’
/home/sdhammo/git/trilinos-github-repo/packages/ifpack2/src/Ifpack2_AdditiveSchwarz_def.hpp:1517:5: required from ‘void Ifpack2::AdditiveSchwarz<MatrixType, LocalInverseType>::setup() [with MatrixType = Tpetra::RowMatrix<double, int, long int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial> >; LocalInverseType = Ifpack2::Preconditioner<double, int, long int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial> >]’
/home/sdhammo/git/trilinos-github-repo/build-devpack-gcc-4.8.4/packages/ifpack2/src/Ifpack2_AdditiveSchwarz_Serial.cpp:61:1: required from here
/home/sdhammo/git/trilinos-github-repo/packages/zoltan2/src/algorithms/order/Zoltan2_AlgNatural.hpp:93:44: error: cannot convert ‘long int*’ to ‘Zoltan2::AlgNatural<Zoltan2::MatrixAdapter<Xpetra::RowMatrix<double, int, long int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial> >, Xpetra::RowMatrix<double, int, long int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial> > > >::lno_t* {aka int*}’ in initialization
lno_t *perm = solution->getPermutation();
^
In file included from /home/sdhammo/git/trilinos-github-repo/packages/zoltan2/src/algorithms/order/Zoltan2_OrderingAlgorithms.hpp:52:0,
from /home/sdhammo/git/trilinos-github-repo/packages/zoltan2/src/problems/Zoltan2_OrderingProblem.hpp:54,
from /home/sdhammo/git/trilinos-github-repo/packages/ifpack2/src/Ifpack2_AdditiveSchwarz_def.hpp:68,
from /home/sdhammo/git/trilinos-github-repo/build-devpack-gcc-4.8.4/packages/ifpack2/src/Ifpack2_AdditiveSchwarz_Serial.cpp:50:
/home/sdhammo/git/trilinos-github-repo/packages/zoltan2/src/algorithms/order/Zoltan2_AlgSortedDegree.hpp: In instantiation of ‘int Zoltan2::AlgSortedDegree<Adapter>::order(const Teuchos::RCP<Zoltan2::OrderingSolution<typename Adapter::lno_t, typename Adapter::gno_t> >&) [with Adapter = Zoltan2::MatrixAdapter<Xpetra::RowMatrix<double, int, long int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial> >, Xpetra::RowMatrix<double, int, long int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial> > >; typename Adapter::gno_t = long int; typename Adapter::lno_t = int]’:
/home/sdhammo/git/trilinos-github-repo/packages/zoltan2/src/problems/Zoltan2_OrderingProblem.hpp:217:7: required from ‘void Zoltan2::OrderingProblem<Adapter>::solve(bool) [with Adapter = Zoltan2::XpetraRowMatrixAdapter<Xpetra::RowMatrix<double, int, long int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial> >, Xpetra::RowMatrix<double, int, long int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial> > >]’
/home/sdhammo/git/trilinos-github-repo/packages/ifpack2/src/Ifpack2_AdditiveSchwarz_def.hpp:1517:5: required from ‘void Ifpack2::AdditiveSchwarz<MatrixType, LocalInverseType>::setup() [with MatrixType = Tpetra::RowMatrix<double, int, long int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial> >; LocalInverseType = Ifpack2::Preconditioner<double, int, long int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial> >]’
/home/sdhammo/git/trilinos-github-repo/build-devpack-gcc-4.8.4/packages/ifpack2/src/Ifpack2_AdditiveSchwarz_Serial.cpp:61:1: required from here
/home/sdhammo/git/trilinos-github-repo/packages/zoltan2/src/algorithms/order/Zoltan2_AlgSortedDegree.hpp:91:44: error: cannot convert ‘long int*’ to ‘Zoltan2::AlgSortedDegree<Zoltan2::MatrixAdapter<Xpetra::RowMatrix<double, int, long int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial> >, Xpetra::RowMatrix<double, int, long int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial> > > >::lno_t* {aka int*}’ in initialization
lno_t *perm = solution->getPermutation();
^
make[2]: *** [packages/ifpack2/src/CMakeFiles/ifpack2.dir/Ifpack2_AdditiveSchwarz_Serial.cpp.o] Error 1
make[1]: *** [packages/ifpack2/src/CMakeFiles/ifpack2.dir/all] Error 2
make: *** [all] Error 2
```
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/93Teuchos on Cray with Intel2016-03-12T13:02:10ZJames WillenbringTeuchos on Cray with Intel*Created by: crtrott*
We are tracking problems all over Trilinos which appear to originate in Teuchos, in particular the Output classes. This manifested in multiple down stream packages as segfaults, which according to some extensive de...*Created by: crtrott*
We are tracking problems all over Trilinos which appear to originate in Teuchos, in particular the Output classes. This manifested in multiple down stream packages as segfaults, which according to some extensive debugging seem to happen due to virtual function pointers which are 0.
The same compiler combination (Intel 15 and GCC 4.9.2) is fine on basic Red Hat system. Currently we suspect a compiler bug or something similar in the cray environment. I will add more details later.
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/82Building Trilinos with Kokkos enabled doesn't work on 32-bit machines2016-03-12T13:06:23ZJames WillenbringBuilding Trilinos with Kokkos enabled doesn't work on 32-bit machines*Created by: BarrySmith*
The PETSc ./configure --download-trilinos=1 --with-cxx-dialect=C++11 --download-boost=1 --download-cmake=1
is failing on one of our machines with error message:
In file included from /sandbox/sarich/petsc/ar...*Created by: BarrySmith*
The PETSc ./configure --download-trilinos=1 --with-cxx-dialect=C++11 --download-boost=1 --download-cmake=1
is failing on one of our machines with error message:
In file included from /sandbox/sarich/petsc/arch-linux2-c-debug/externalpackages/git.trilinos/packages/kokkos/core/src/Kokkos_Atomic.hpp:208:0,
from /sandbox/sarich/petsc/arch-linux2-c-debug/externalpackages/git.trilinos/packages/kokkos/core/src/impl/Kokkos_AllocationTracker.cpp:50:
/sandbox/sarich/petsc/arch-linux2-c-debug/externalpackages/git.trilinos/packages/kokkos/core/src/impl/Kokkos_Atomic_Generic.hpp: In instantiation of ‘T Kokkos::Impl::atomic_fetch_oper(const Oper&, volatile T_, typename Kokkos::Impl::enable_if<((sizeof (T) != sizeof (int)) && (sizeof (T) == sizeof (long long unsigned int))), const T>::type) [with Oper = Kokkos::Impl::AndOper<long long unsigned int, const long long unsigned int>; T = long long unsigned int; typename Kokkos::Impl::enable_if<((sizeof (T) != sizeof (int)) && (sizeof (T) == sizeof (long long unsigned int))), const T>::type = const long long unsigned int]’:
/sandbox/sarich/petsc/arch-linux2-c-debug/externalpackages/git.trilinos/packages/kokkos/core/src/impl/Kokkos_Atomic_Generic.hpp:296:69: required from ‘T Kokkos::atomic_fetch_and(volatile T_, T) [with T = long long unsigned int]’
/sandbox/sarich/petsc/arch-linux2-c-debug/externalpackages/git.trilinos/packages/kokkos/core/src/impl/Kokkos_AllocationTracker.cpp:211:64: required from ‘bool Kokkos::Impl::{anonymous}::Bitset<NumBlocks>::reset(int) [with int NumBlocks = 15]’
/sandbox/sarich/petsc/arch-linux2-c-debug/externalpackages/git.trilinos/packages/kokkos/core/src/impl/Kokkos_AllocationTracker.cpp:296:30: required from here
/sandbox/sarich/petsc/arch-linux2-c-debug/externalpackages/git.trilinos/packages/kokkos/core/src/impl/Kokkos_Atomic_Compare_Exchange_Strong.hpp:207:3: note: template<class T> T Kokkos::atomic_compare_exchange(volatile T_, T, typename Kokkos::Impl::enable_if<((sizeof (T) != 4) && (sizeof (T) != 8)), const T>::type&)
T atomic_compare_exchange( volatile T \* const dest , const T compare ,
^
/sandbox/sarich/petsc/arch-linux2-c-debug/externalpackages/git.trilinos/packages/kokkos/core/src/impl/Kokkos_Atomic_Compare_Exchange_Strong.hpp:207:3: note: template argument deduction/substitution failed:
/sandbox/sarich/petsc/arch-linux2-c-debug/externalpackages/git.trilinos/packages/kokkos/core/src/impl/Kokkos_Atomic_Compare_Exchange_Strong.hpp: In substitution of ‘template<class T> T Kokkos::atomic_compare_exchange(volatile T_, T, typename Kokkos::Impl::enable_if<((sizeof (T) != 4) && (sizeof (T) != 8)), const T>::type&) [with T = long long unsigned int]’:
/sandbox/sarich/petsc/arch-linux2-c-debug/externalpackages/git.trilinos/packages/kokkos/core/src/impl/Kokkos_Atomic_Generic.hpp:144:103: required from ‘T Kokkos::Impl::atomic_fetch_oper(const Oper&, volatile T_, typename Kokkos::Impl::enable_if<((sizeof (T) != sizeof (int)) && (sizeof (T) == sizeof (long long unsigned int))), const T>::type) [with Oper = Kokkos::Impl::AndOper<long long unsigned int, const long long unsigned int>; T = long long unsigned int; typename Kokkos::Impl::enable_if<((sizeof (T) != sizeof (int)) && (sizeof (T) == sizeof (long long unsigned int))), const T>::type = const long long unsigned int]’
/sandbox/sarich/petsc/arch-linux2-c-debug/externalpackages/git.trilinos/packages/kokkos/core/src/impl/Kokkos_Atomic_Generic.hpp:296:69: required from ‘T Kokkos::atomic_fetch_and(volatile T_, T) [with T = long long unsigned int]’
/sandbox/sarich/petsc/arch-linux2-c-debug/externalpackages/git.trilinos/packages/kokkos/core/src/impl/Kokkos_AllocationTracker.cpp:211:64: required from ‘bool Kokkos::Impl::{anonymous}::Bitset<NumBlocks>::reset(int) [with int NumBlocks = 15]’
/sandbox/sarich/petsc/arch-linux2-c-debug/externalpackages/git.trilinos/packages/kokkos/core/src/impl/Kokkos_AllocationTracker.cpp:296:30: required from here
/sandbox/sarich/petsc/arch-linux2-c-debug/externalpackages/git.trilinos/packages/kokkos/core/src/impl/Kokkos_Atomic_Compare_Exchange_Strong.hpp:207:3: error: invalid use of incomplete type ‘struct Kokkos::Impl::enable_if<false, const long long unsigned int>’
In file included from /sandbox/sarich/petsc/arch-linux2-c-debug/externalpackages/git.trilinos/packages/kokkos/core/src/Kokkos_MemoryTraits.hpp:47:0,
from /sandbox/sarich/petsc/arch-linux2-c-debug/externalpackages/git.trilinos/packages/kokkos/core/src/Kokkos_HostSpace.hpp:53,
from /sandbox/sarich/petsc/arch-linux2-c-debug/externalpackages/git.trilinos/packages/kokkos/core/src/Kokkos_Atomic.hpp:71,
from /sandbox/sarich/petsc/arch-linux2-c-debug/externalpackages/git.trilinos/packages/kokkos/core/src/impl/Kokkos_AllocationTracker.cpp:50:
/sandbox/sarich/petsc/arch-linux2-c-debug/externalpackages/git.trilinos/packages/kokkos/core/src/impl/Kokkos_Traits.hpp:201:8: error: declaration of ‘struct Kokkos::Impl::enable_if<false, const long long unsigned int>’
struct enable_if ;
^
Full output from configure/build is attached
[configure.txt](https://github.com/trilinos/Trilinos/files/95073/configure.txt)
Barry
@sarich
@balay@mcs.anl.gov
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/149boostlib defines not being respected.2016-03-16T15:08:31ZJames Willenbringboostlib defines not being respected.*Created by: bathmatt*
On hansen I'm working on building trilinos and I have set every thing I can think of to set the boost library paths.
dir-debug (gcc) 20 $ grep BoostL configure-drekar.sh
-D TPL_ENABLE_BoostLib=ON \
-D TPL_BoostLi...*Created by: bathmatt*
On hansen I'm working on building trilinos and I have set every thing I can think of to set the boost library paths.
dir-debug (gcc) 20 $ grep BoostL configure-drekar.sh
-D TPL_ENABLE_BoostLib=ON \
-D TPL_BoostLib_INCLUDE_DIRS:FILEPATH="${BOOST_BASE_DIR}/include" \
-D TPL_BoostLib_LIBRARY_DIRS:FILEPATH="${BOOST_BASE_DIR}/lib" \
-D TPL_BoostLib_LIBRARIES="${BOOST_BASE_DIR}/lib/libboost_program_options.so;${BOOST_BASE_DIR}/lib/libboost_system.so" \
However, cmake ignores this and just uses -lboost_system as shown below. This happens on lots of different systems, hansen, chama are just two I can think of. I have a long email thread with Ross but that hit a dead end.
[config.out.txt](https://github.com/trilinos/Trilinos/files/138446/config.out.txt)
/home/projects/x86-64-haswell/openmpi/1.10.0/gnu/4.9.3/bin/mpicxx -std=c++11 -fopenmp -g -O0 CMakeFiles/STKClassic_stk_algsup_unit_tests.dir/UnitTestAlgorithmRunner.cpp.o CMakeFiles/STKClassic_stk_algsup_unit_tests.dir/UnitTestCudaMgr.cpp.o CMakeFiles/STKClassic_stk_algsup_unit_tests.dir/UnitTestMain.cpp.o CMakeFiles/STKClassic_stk_algsup_unit_tests.dir/UnitTest_helpers.cpp.o -o STKClassic_stk_algsup_unit_tests.exe -rdynamic ../stk_algsup/libstkclassic_algsup.a ../../stk_mesh/stk_mesh/fixtures/libstkclassic_mesh_fixtures.a ../../stk_mesh/stk_mesh/fem/libstkclassic_mesh_fem.a ../../stk_mesh/stk_mesh/base/libstkclassic_mesh_base.a ../../stk_util/stk_util/unit_test_support/libstkclassic_util_unit_test_support.a ../../stk_util/stk_util/parallel/libstkclassic_util_parallel.a ../../stk_util/stk_util/diag/libstkclassic_util_diag.a ../../stk_util/stk_util/environment/libstkclassic_util_env.a ../../stk_util/stk_util/util/libstkclassic_util_util.a ../../../../seacas/libraries/exodus/cbind/libexodus.a -lboost_program_options -lboost_system ../../../../fei/support-Trilinos/libfei_trilinos.a ../../../../fei/base/libfei_base.a ../../../../belos/tpetra/src/libbelostpetra.a ../../../../belos/epetra/src/libbelosepetra.a ../../../../belos/src/libbelos.a ../../../../ml/src/libml.a ../../../../galeri/src-epetra/libgaleri-epetra.a ../../../../tpetra/core/ext/libtpetraext.a ../../../../tpetra/core/inout/libtpetrainout.a ../../../../tpetra/core/src/libtpetra.a ../../../../tpetra/kernels/src/libtpetrakernels.a ../../../../kokkos/algorithms/src/libkokkosalgorithms.a ../../../../kokkos/containers/src/libkokkoscontainers.a ../../../../tpetra/classic/LinAlg/libtpetraclassiclinalg.a ../../../../tpetra/classic/NodeAPI/libtpetraclassicnodeapi.a ../../../../tpetra/classic/src/libtpetraclassic.a ../../../../ifpack/src/libifpack.a ../../../../amesos/src/libamesos.a ../../../../epetraext/src/libepetraext.a ../../../../seacas/libraries/ioss/src/init/libIonit.a ../../../../seacas/libraries/ioss/src/transform/libIotr.a ../../../../seacas/libraries/ioss/src/heartbeat/libIohb.a ../../../../seacas/libraries/ioss/src/generated/libIogn.a ../../../../seacas/libraries/ioss/src/pamgen/libIopg.a ../../../../seacas/libraries/ioss/src/exo_fac/libIoexo_fac.a ../../../../seacas/libraries/ioss/src/exo_par/libIopx.a ../../../../seacas/libraries/ioss/src/exo_fpp/libIofx.a ../../../../seacas/libraries/ioss/src/exodus/libIoex.a ../../../../seacas/libraries/ioss/src/libIoss.a ../../../../seacas/libraries/exodus/cbind/libexodus.a -Wl,-Bstatic -lnetcdf -Wl,-Bdynamic -L/home/projects/x86-64-haswell-nvidia/netcdf-exo/4.3.3.1/openmpi/1.10.0/gcc/4.8.4/cuda/7.5.7/lib -lnetcdf -L/home/projects/x86-64-haswell-nvidia/hdf5/1.8.15/openmpi/1.10.0/gcc/4.8.4/cuda/7.5.7/lib -lhdf5_hl -lhdf5 -lz -ldl ../../../../pamgen/src/libpamgen_extras.a ../../../../pamgen/src/libpamgen.a ../../../../aztecoo/src/libaztecoo.a ../../../../triutils/src/libtriutils.a ../../../../epetra/src/libepetra.a ../../../../shards/src/libshards.a ../../../../zoltan/src/libzoltan.a -lm ../../../../sacado/src/libsacado.a ../../../../teuchos/kokkoscomm/src/libteuchoskokkoscomm.a ../../../../teuchos/kokkoscompat/src/libteuchoskokkoscompat.a ../../../../teuchos/remainder/src/libteuchosremainder.a ../../../../teuchos/numerics/src/libteuchosnumerics.a -L/home/projects/x86-64-haswell/lapack/3.5.0/gcc/4.8.4 -llapack -L/home/projects/x86-64-haswell/blas/20150602/gcc/4.8.4 -lblas ../../../../teuchos/comm/src/libteuchoscomm.a ../../../../teuchos/parameterlist/src/libteuchosparameterlist.a ../../../../teuchos/core/src/libteuchoscore.a -lboost_program_options -lboost_system ../../../../kokkos/core/src/libkokkoscore.a -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lgfortran -lquadmath
I even went so far as to edit the cmake dependencies file
-bash-4.1$ git diff
diff --git a/packages/stk/stk_classic/cmake/Dependencies.cmake b/packages/stk/stk_classic/cmake/Dependencies.cmake
index 0c16502..b7cd94d 100644
--- a/packages/stk/stk_classic/cmake/Dependencies.cmake
+++ b/packages/stk/stk_classic/cmake/Dependencies.cmake
@@ -4,5 +4,5 @@ SET(TEST_REQUIRED_DEP_PACKAGES SEACASExodus)
SET(TEST_OPTIONAL_DEP_PACKAGES)
SET(LIB_REQUIRED_DEP_TPLS Boost BoostLib)
SET(LIB_OPTIONAL_DEP_TPLS OpenNURBS)
-SET(TEST_REQUIRED_DEP_TPLS)
+SET(TEST_REQUIRED_DEP_TPLS BoostLib)
SET(TEST_OPTIONAL_DEP_TPLS gtest)
-bash-4.1$
dir-debug (gcc) 23 $ cmake --version
cmake version 3.3.2
CMake suite maintained and supported by Kitware (kitware.com/cmake).
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/216Unable to Specify --map-by numa:PE=4 in Trilinos Configuration for MPI Tests2016-03-16T20:06:58ZJames WillenbringUnable to Specify --map-by numa:PE=4 in Trilinos Configuration for MPI Tests*Created by: nmhamster*
@bartlettroscoe I have been specifying `-D MPI_EXEC_POST_NUMPROCS_FLAGS:STRING="--map-by numa:PE=4"` in my Trilinos configuration but it appears that this is being passed to `mpirun` or `mpiexec` as a single argu...*Created by: nmhamster*
@bartlettroscoe I have been specifying `-D MPI_EXEC_POST_NUMPROCS_FLAGS:STRING="--map-by numa:PE=4"` in my Trilinos configuration but it appears that this is being passed to `mpirun` or `mpiexec` as a single argument (i.e. not two arguments).
I'd like to be able to run the Trilinos test in a sensible binding mode for OpenMP on `hansen`, `shiller` and `shepard`.
So far I have backed off to `--bind-to-socket` (a single argument) but it would be better to not have MPI ranks conflicting if we can avoid that (and use the true mapping option).
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/219Compilation error for missing operator >= for Kokkos_Views when called by Tp...2016-03-16T23:35:43ZJames WillenbringCompilation error for missing operator >= for Kokkos_Views when called by Tpetra_MultiVector*Created by: mperego*
It's probably related with new Kokkos Views. See my configuration script and error that I get.
Thanks!
Mauro
[do-trilinos-experimental.txt](https://github.com/trilinos/Trilinos/files/176892/do-trilinos-experimen...*Created by: mperego*
It's probably related with new Kokkos Views. See my configuration script and error that I get.
Thanks!
Mauro
[do-trilinos-experimental.txt](https://github.com/trilinos/Trilinos/files/176892/do-trilinos-experimental.txt)
[error.txt](https://github.com/trilinos/Trilinos/files/176893/error.txt)
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/203TriUtils build failure with Teuchos disabled2016-03-17T13:25:42ZJames WillenbringTriUtils build failure with Teuchos disabled*Created by: jwillenbring*
TriUtils currently has no stated dependence on Teuchos. It does have a stated dependence on Epetra. Epetra has an optional dependence on Teuchos, so Teuchos is typically available. On Sept 30, TriUtils was mad...*Created by: jwillenbring*
TriUtils currently has no stated dependence on Teuchos. It does have a stated dependence on Epetra. Epetra has an optional dependence on Teuchos, so Teuchos is typically available. On Sept 30, TriUtils was made dependent on Teuchos, although the dependencies file was not updated.
commit 60dd431da705a59158cecc30055b62b593024824
Author: Nico Schlömer nico.schloemer@gmail.com
Date: Tue Sep 30 11:39:16 2014 +0200
```
[TriUtils] char *data_file ==> const char *data_file
Using a `const char*` as function argument allows char literals
as arguments.
Besides this change, this commit contains a bunch of indentation corrections
and replacement of conditional `exit(...)` calls by their TEUCHOS_EXCEPT*
equivalents.
```
Basically the options are to make TriUtils depend on Teuchos, or make the Teuchos calls conditional.
@hkthorn - this is affecting the no C++11 nightly build.
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/205Tpetra::CrsGraph::findLocalIndex: Port to use Kokkos2016-03-17T13:36:36ZJames WillenbringTpetra::CrsGraph::findLocalIndex: Port to use Kokkos*Created by: mhoemmen*
@trilinos/tpetra @amklinv
This blocks #41.
It may also help #118.
Tpetra::CrsGraph::findLocalIndex takes the column indices in a row of a CrsGraph / CrsMatrix, and a column index. If the column index exists ...*Created by: mhoemmen*
@trilinos/tpetra @amklinv
This blocks #41.
It may also help #118.
Tpetra::CrsGraph::findLocalIndex takes the column indices in a row of a CrsGraph / CrsMatrix, and a column index. If the column index exists in the row, it returns the relative offset of that column index. Otherwise, it returns a flag value. There are other details (e.g., the search hint) but that's the essence.
We want to make this fully Kokkos-ized -- we want this functionality, marked as a Kokkos device function, so that we can use it in Kokkos::parallel_*.
There is nothing in this method that requires knowing anything about a sparse graph or matrix! It's just search in an array. Thus, if we want to Kokkos-ize this fully, it shouldn't even be an instance method of KokkosSparse::CrsMatrix, nor does it need to take the matrix as an input argument. I don't want to add more instance methods to classes, especially not public ones.
Doing this will get us most of the way to finishing #41. It should also start to address #118, in that it will help us change Tpetra so that dispatch from replace / sumInto to the various storage options will happen at the top level, making the lower-level search code less general and therefore possibly faster.
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/220Duplicate lib refs if one uses cmake Trilinos_ENABLE_INSTALL_CMAKE_CONFIG_FIL...2016-03-17T14:58:23ZJames WillenbringDuplicate lib refs if one uses cmake Trilinos_ENABLE_INSTALL_CMAKE_CONFIG_FILES:BOOL=ON*Created by: bathmatt*
First this is a low priority ticket, but when I do an install of trilinos and export the cmake stuff, and use it in my application I get a lot of duplicate lib paths, roughly 30 per lib. I've attached the link li...*Created by: bathmatt*
First this is a low priority ticket, but when I do an install of trilinos and export the cmake stuff, and use it in my application I get a lot of duplicate lib paths, roughly 30 per lib. I've attached the link line in the file. If there could be some consolidation that would be great, it is hard to see the tree from the forest.
[link.txt](https://github.com/trilinos/Trilinos/files/176948/link.txt)
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/221Missing Sacado test dependency2016-03-17T17:39:22ZJames WillenbringMissing Sacado test dependency*Created by: jwillenbring*
@etphipp
When building Sacado for a nightly no-C++11 build with optional packages off, the following error occurred:
/jenkins/slave/workspace/Trilinos_XYCE_MPI/MPI_OPT_DEV_XYCE/Trilinos/packages/sacado/test...*Created by: jwillenbring*
@etphipp
When building Sacado for a nightly no-C++11 build with optional packages off, the following error occurred:
/jenkins/slave/workspace/Trilinos_XYCE_MPI/MPI_OPT_DEV_XYCE/Trilinos/packages/sacado/test/UnitTests/ConversionTests.cpp:30:39: error: Teuchos_UnitTestHarness.hpp: No such file or directory
/jenkins/slave/workspace/Trilinos_XYCE_MPI/MPI_OPT_DEV_XYCE/Trilinos/packages/sacado/test/UnitTests/ConversionTests.cpp:31:42: error: Teuchos_UnitTestRepository.hpp: No such file or directory
/jenkins/slave/workspace/Trilinos_XYCE_MPI/MPI_OPT_DEV_XYCE/Trilinos/packages/sacado/test/UnitTests/ConversionTests.cpp:32:40: error: Teuchos_GlobalMPISession.hpp: No such file or directory
/jenkins/slave/workspace/Trilinos_XYCE_MPI/MPI_OPT_DEV_XYCE/Trilinos/packages/sacado/test/UnitTests/ConversionTests.cpp:33:38: error: Teuchos_TestingHelpers.hpp: No such file or directory
/jenkins/slave/workspace/Trilinos_XYCE_MPI/MPI_OPT_DEV_XYCE/Trilinos/packages/sacado/test/UnitTests/ConversionTests.cpp:53: error: expected constructor, destructor, or type conversion before ‘(’ token
/jenkins/slave/workspace/Trilinos_XYCE_MPI/MPI_OPT_DEV_XYCE/Trilinos/packages/sacado/test/UnitTests/ConversionTests.cpp:70: error: template declaration of ‘bool test_ad_conversions’
/jenkins/slave/workspace/Trilinos_XYCE_MPI/MPI_OPT_DEV_XYCE/Trilinos/packages/sacado/test/UnitTests/ConversionTests.cpp:70: error: ‘Teuchos’ has not been declared
...
The full context can be seen here:
http://testing.sandia.gov/cdash/viewBuildError.php?buildid=2380725
I have a simple fix that I have tested which is to add TeuchosCore as a required test dependence for Sacado. I did not investigate if it would be possible to refactor the testing to optionally use Teuchos. It seems very likely that this case isn't commonly exercised, as Teuchos is an optional library dependence for Sacado, and Teuchos is included in the vast majority of Trilinos builds.
I plan to issue a pull request for this simple fix later today.
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/206KokkosContainers_UnitTest_MPI_1 Time Out on Shepard using GCC 4.8.42016-03-18T08:49:54ZJames WillenbringKokkosContainers_UnitTest_MPI_1 Time Out on Shepard using GCC 4.8.4*Created by: nmhamster*
KokkosContainers_UnitTest_MPI_1 test is timing out on Shepard using GCC 4.8.4.
To reproduce:
- `module load devpack/openmpi/1.10.0/gcc/4.8.4/cuda/7.5.18`
- Configure Trilinos
- `make`
- `make test`
*Created by: nmhamster*
KokkosContainers_UnitTest_MPI_1 test is timing out on Shepard using GCC 4.8.4.
To reproduce:
- `module load devpack/openmpi/1.10.0/gcc/4.8.4/cuda/7.5.18`
- Configure Trilinos
- `make`
- `make test`
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/233KokkosKernels: Make sure sparse mat-vec instantiations use same OFFSET_TYPE t...2016-03-19T07:33:46ZJames WillenbringKokkosKernels: Make sure sparse mat-vec instantiations use same OFFSET_TYPE that Tpetra does*Created by: mhoemmen*
@trilinos/tpetra KokkosKernels does explicit instantiations (not through the ETI system -- it's a different approach that works whether or not ETI is enabled) for the sparse matrix-vector multiply kernels that Tpe...*Created by: mhoemmen*
@trilinos/tpetra KokkosKernels does explicit instantiations (not through the ETI system -- it's a different approach that works whether or not ETI is enabled) for the sparse matrix-vector multiply kernels that Tpetra and downstream packages use. The instantiations specify the offset type (type of the `ptr` array in the CSR `ptr`, `ind`, `val` triple) explicitly as `size_t` for all device types, including for CUDA. However, with CUDA, Kokkos may use a different size_type than `size_t` -- in particular, it may use a 32-bit integer (such as `int`). This means that the instantiations are getting built, but not getting used!
Fixing this may not actually fix #226, but certainly goes along with its spirit.
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/231KokkosKernels: Don't need both CudaSpace & CudaUVMSpace instantiations2016-03-19T08:19:49ZJames WillenbringKokkosKernels: Don't need both CudaSpace & CudaUVMSpace instantiations*Created by: mhoemmen*
@trilinos/tpetra
See #226. CUDA instantiations of sparse matrix-vector multiply kernel take a long time. I found that the kernels used by Tpetra for iterative solvers (sparse mat-vec and (multi)vector operatio...*Created by: mhoemmen*
@trilinos/tpetra
See #226. CUDA instantiations of sparse matrix-vector multiply kernel take a long time. I found that the kernels used by Tpetra for iterative solvers (sparse mat-vec and (multi)vector operations) are getting instantiated for both `Device<Cuda, CudaSpace>` and `Device<Cuda, CudaUVMSpace>`. Tpetra and downstream packages only need the latter. So, it's building twice as much code as needed.
KokkosKernels instantiates these kernels in a different way than Trilinos' ETI. With KokkosKernels, one can use template parameter combinations that haven't been instantiated. Thus, getting rid of instantiations that Tpetra and downstream packages don't use won't break any tests, examples, or downstream code.
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/208build of Kokkos_Sparse_MV_impl_spmv_Serial.cpp.o fails if you use nvcc and ha...2016-03-19T08:19:49ZJames Willenbringbuild of Kokkos_Sparse_MV_impl_spmv_Serial.cpp.o fails if you use nvcc and have cuda disabled*Created by: bathmatt*
If I don't configure with cuda but still have OMPI_CXX=nvcc_wrapper I get the following. This is on hansen
[ 66%] Building CXX object packages/tpetra/kernels/src/CMakeFiles/tpetrakernels.dir/impl/Kokkos_Sparse_MV...*Created by: bathmatt*
If I don't configure with cuda but still have OMPI_CXX=nvcc_wrapper I get the following. This is on hansen
[ 66%] Building CXX object packages/tpetra/kernels/src/CMakeFiles/tpetrakernels.dir/impl/Kokkos_Sparse_MV_impl_spmv_Serial.cpp.o
/home/mbetten/Trilinos/Trilinos/packages/tpetra/kernels/src/impl/Kokkos_Sparse_impl_spmv.hpp(885): error: namespace "Kokkos" has no member "shfl_down"
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/232KokkosKernels: Make sure sparse mat-vec instantiations use same OFFSET_TYPE t...2016-03-19T08:50:26ZJames WillenbringKokkosKernels: Make sure sparse mat-vec instantiations use same OFFSET_TYPE that Tpetra does*Created by: mhoemmen*
@trilinos/tpetra KokkosKernels does explicit instantiations (not through the ETI system -- it's a different approach that works whether or not ETI is enabled) for the sparse matrix-vector multiply kernels that Tpe...*Created by: mhoemmen*
@trilinos/tpetra KokkosKernels does explicit instantiations (not through the ETI system -- it's a different approach that works whether or not ETI is enabled) for the sparse matrix-vector multiply kernels that Tpetra and downstream packages use. The instantiations specify the offset type (type of the `ptr` array in the CSR `ptr`, `ind`, `val` triple) explicitly as `size_t` for all device types, including for CUDA. However, with CUDA, Kokkos may use a different size_type than `size_t` -- in particular, it may use a 32-bit integer (such as `int`). This means that the instantiations are getting built, but not getting used!
Fixing this may not actually fix #226, but certainly goes along with its spirit.