Commit 56a7f9b7 authored by Mark Hoemmen's avatar Mark Hoemmen
Browse files

Moved KokkosLinAlg subpackage from Kokkos into Tpetra as TpetraKernels.

I moved the KokkosLinAlg subpackage of Kokkos into Tpetra, and renamed
it "TpetraKernels".  This move will facilitate moving Kokkos from the
Trilinos repository into its own Github repository.  I renamed it from
"LinAlg" because all of Tpetra implements "linear algebra."  The name
"Kernels" better suggests the contents: single-MPI-process,
thread-parallel computational kernels.

I also fixed downstream packages that depended on KokkosLinAlg.  This
includes both the package name (KokkosLinAlg -> TpetraKernels) and
various library names.

This commit mainly affects Kokkos and Tpetra.  I also had to make
minor changes to Isorropia, ShyLu, Stokhos, TrilinosCouplings, and
Xpetra.  In particular, Xpetra's "FakeKokkos" (KokkosClassic
replacement headers for when building with Tpetra disabled) caused
some bizarre build errors until I finally figured out that Xpetra was
using "Xpetra_ENABLE_Kokkos" to key on whether to use FakeKokkos'
headers.  This manifested as Tpetra files not getting the contents of
TpetraClassic files (like Kokkos_ConfigDefs.hpp), but only when
building in Xpetra.  I fixed this by making Xpetra key on
Xpetra_ENABLE_Tpetra, since the former KokkosClassic now lives in
Tpetra.

NOTE: The Kokkos refactor version of MueLu does not currently build
with ETI enabled, because of some missing explicit instantiations.
Here is the error message I get:

Linking CXX executable MueLu_Challenge_XML.exe
../../../src/libmuelu.so.11.13: undefined reference to `Ifpack2::AdditiveSchwarz<Tpetra::RowMatrix<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::OpenMP> >, Ifpack2::Preconditioner<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::OpenMP> > >::AdditiveSchwarz(Teuchos::RCP<Tpetra::RowMatrix<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::OpenMP> > const> const&, int)'
../../../src/libmuelu.so.11.13: undefined reference to `Ifpack2::Details::OneLevelFactory<Tpetra::RowMatrix<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::OpenMP> > >::create(std::string const&, Teuchos::RCP<Tpetra::RowMatrix<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::OpenMP> > const> const&) const'
../../../src/libmuelu.so.11.13: undefined reference to `Ifpack2::Chebyshev<Tpetra::RowMatrix<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::OpenMP> > >::getLambdaMaxForApply() const'
../../../src/libmuelu.so.11.13: undefined reference to `Ifpack2::Hiptmair<Tpetra::CrsMatrix<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::OpenMP> > >::Hiptmair(Teuchos::RCP<Tpetra::RowMatrix<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::OpenMP> > const> const&, Teuchos::RCP<Tpetra::RowMatrix<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::OpenMP> > const> const&, Teuchos::RCP<Tpetra::RowMatrix<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::OpenMP> > const> const&)'
../../../src/libmuelu.so.11.13: undefined reference to `Ifpack2::Krylov<Tpetra::RowMatrix<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::OpenMP> > >::Krylov(Teuchos::RCP<Tpetra::RowMatrix<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::OpenMP> > const> const&)'
collect2: error: ld returned 1 exit status

Note that KokkosDeviceWrapperNode<Kokkos::OpenMP> is NOT the default
Node type in this build; the default Node type is
KokkosDeviceWrapperNode<Kokkos::Serial>.  As far as I can tell,
Ifpack2 only does ETI for the default Node type.  It could be that
MueLu is explicitly requesting ETI for a NON-default Node type, or it
could be that I messed up somewhere with the macros.  I'll have to
work a little bit harder at this, but I think it's OK to finish the
subpackage migration process first.
parent e17acde0
NOTE: It is unlikely that the raw build scripts in this directory
still work. For now, please use the CMake build system to build these
examples.
......@@ -2,7 +2,7 @@ SET(LIB_REQUIRED_DEP_PACKAGES)
SET(LIB_OPTIONAL_DEP_PACKAGES
EpetraExt Isorropia Amesos AztecOO Belos Ifpack ML NOX Zoltan STKClassic Stokhos)
SET(TEST_REQUIRED_DEP_PACKAGES)
SET(TEST_OPTIONAL_DEP_PACKAGES Amesos Isorropia Epetra EpetraExt Ifpack Intrepid Pamgen AztecOO ML Zoltan STKClassic Teko Tpetra MueLu KokkosCore KokkosLinAlg KokkosCompat KokkosContainers KokkosMpiComm Stokhos)
SET(TEST_OPTIONAL_DEP_PACKAGES Amesos Isorropia Epetra EpetraExt Ifpack Intrepid Pamgen AztecOO ML Zoltan STKClassic Teko TpetraKernels Tpetra MueLu KokkosCore KokkosCompat KokkosContainers KokkosMpiComm Stokhos)
SET(LIB_REQUIRED_DEP_TPLS)
SET(LIB_OPTIONAL_DEP_TPLS)
SET(TEST_REQUIRED_DEP_TPLS)
......
......@@ -30,7 +30,7 @@ IF (TrilinosCouplings_ENABLE_Tpetra AND Tpetra_Have_Kokkos_Refactor)
CUDA_INCLUDE_DIRECTORIES( ${Tpetra_INCLUDE_DIRS} )
CUDA_INCLUDE_DIRECTORIES( ${KokkosCore_INCLUDE_DIRS} )
CUDA_INCLUDE_DIRECTORIES( ${KokkosContainers_INCLUDE_DIRS} )
CUDA_INCLUDE_DIRECTORIES( ${KokkosLinAlg_INCLUDE_DIRS} )
CUDA_INCLUDE_DIRECTORIES( ${TpetraKernels_INCLUDE_DIRS} )
CUDA_INCLUDE_DIRECTORIES( ${KokkosCompat_INCLUDE_DIRS} )
IF (TrilinosCouplings_ENABLE_MueLu)
CUDA_INCLUDE_DIRECTORIES( ${MueLu_INCLUDE_DIRS} )
......@@ -52,7 +52,7 @@ IF (TrilinosCouplings_ENABLE_Tpetra AND Tpetra_Have_Kokkos_Refactor)
IF( TrilinosCouplings_ENABLE_Stokhos AND
Stokhos_ENABLE_Sacado AND
Stokhos_ENABLE_KokkosCore AND
Stokhos_ENABLE_KokkosLinAlg AND
Stokhos_ENABLE_TpetraKernels AND
Stokhos_ENABLE_Tpetra )
SET( SOURCES fenl_ensemble.hpp
......
......@@ -38,7 +38,15 @@ ENDIF()
#SET_AND_INC_DIRS(DIR ${CMAKE_CURRENT_SOURCE_DIR})
IF (NOT ${PROJECT_NAME}_ENABLE_Kokkos)
# mfh 18 Dec 2014: The "real" Kokkos(Classic) is now the TpetraClassic
# subpackage of Tpetra. Thus, we have to test whether Tpetra is
# enabled, not whether Kokkos is enabled. Furthermore, it wasn't
# previously correct to test if Kokkos was enabled, since Kokkos could
# have been enabled without the KokkosClassic subpackage having been
# enabled.
#
#IF (NOT ${PROJECT_NAME}_ENABLE_Kokkos)
IF (NOT ${PROJECT_NAME}_ENABLE_Tpetra)
SET_AND_INC_DIRS(DIR ${CMAKE_CURRENT_SOURCE_DIR}/FakeKokkos)
APPEND_GLOB(HEADERS FakeKokkos/*.hpp)
ENDIF()
......
......@@ -17,8 +17,6 @@
# ${TEUCHOS_STD_UNIT_TEST_MAIN}
# COMM serial mpi
# STANDARD_PASS_OUTPUT
# #TODO
# # DEPLIBS kokkos kokkoslinalg kokkosnodeapi
# )
IF (NOT Xpetra_Epetra_NO_32BIT_GLOBAL_INDICES)
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment