Skip to content
  • Mark Hoemmen's avatar
    Moved KokkosLinAlg subpackage from Kokkos into Tpetra as TpetraKernels. · 56a7f9b7
    Mark Hoemmen authored
    I moved the KokkosLinAlg subpackage of Kokkos into Tpetra, and renamed
    it "TpetraKernels".  This move will facilitate moving Kokkos from the
    Trilinos repository into its own Github repository.  I renamed it from
    "LinAlg" because all of Tpetra implements "linear algebra."  The name
    "Kernels" better suggests the contents: single-MPI-process,
    thread-parallel computational kernels.
    
    I also fixed downstream packages that depended on KokkosLinAlg.  This
    includes both the package name (KokkosLinAlg -> TpetraKernels) and
    various library names.
    
    This commit mainly affects Kokkos and Tpetra.  I also had to make
    minor changes to Isorropia, ShyLu, Stokhos, TrilinosCouplings, and
    Xpetra.  In particular, Xpetra's "FakeKokkos" (KokkosClassic
    replacement headers for when building with Tpetra disabled) caused
    some bizarre build errors until I finally figured out that Xpetra was
    using "Xpetra_ENABLE_Kokkos" to key on whether to use FakeKokkos'
    headers.  This manifested as Tpetra files not getting the contents of
    TpetraClassic files (like Kokkos_ConfigDefs.hpp), but only when
    building in Xpetra.  I fixed this by making Xpetra key on
    Xpetra_ENABLE_Tpetra, since the former KokkosClassic now lives in
    Tpetra.
    
    NOTE: The Kokkos refactor version of MueLu does not currently build
    with ETI enabled, because of some missing explicit instantiations.
    Here is the error message I get:
    
    Linking CXX executable MueLu_Challenge_XML.exe
    ../../../src/libmuelu.so.11.13: undefined reference to `Ifpack2::AdditiveSchwarz<Tpetra::RowMatrix<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::OpenMP> >, Ifpack2::Preconditioner<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::OpenMP> > >::AdditiveSchwarz(Teuchos::RCP<Tpetra::RowMatrix<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::OpenMP> > const> const&, int)'
    ../../../src/libmuelu.so.11.13: undefined reference to `Ifpack2::Details::OneLevelFactory<Tpetra::RowMatrix<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::OpenMP> > >::create(std::string const&, Teuchos::RCP<Tpetra::RowMatrix<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::OpenMP> > const> const&) const'
    ../../../src/libmuelu.so.11.13: undefined reference to `Ifpack2::Chebyshev<Tpetra::RowMatrix<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::OpenMP> > >::getLambdaMaxForApply() const'
    ../../../src/libmuelu.so.11.13: undefined reference to `Ifpack2::Hiptmair<Tpetra::CrsMatrix<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::OpenMP> > >::Hiptmair(Teuchos::RCP<Tpetra::RowMatrix<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::OpenMP> > const> const&, Teuchos::RCP<Tpetra::RowMatrix<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::OpenMP> > const> const&, Teuchos::RCP<Tpetra::RowMatrix<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::OpenMP> > const> const&)'
    ../../../src/libmuelu.so.11.13: undefined reference to `Ifpack2::Krylov<Tpetra::RowMatrix<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::OpenMP> > >::Krylov(Teuchos::RCP<Tpetra::RowMatrix<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::OpenMP> > const> const&)'
    collect2: error: ld returned 1 exit status
    
    Note that KokkosDeviceWrapperNode<Kokkos::OpenMP> is NOT the default
    Node type in this build; the default Node type is
    KokkosDeviceWrapperNode<Kokkos::Serial>.  As far as I can tell,
    Ifpack2 only does ETI for the default Node type.  It could be that
    MueLu is explicitly requesting ETI for a NON-default Node type, or it
    could be that I messed up somewhere with the macros.  I'll have to
    work a little bit harder at this, but I think it's OK to finish the
    subpackage migration process first.
    56a7f9b7