Trilinos issueshttps://gitlab.osti.gov/jmwille/Trilinos/-/issues2017-11-28T20:28:09Zhttps://gitlab.osti.gov/jmwille/Trilinos/-/issues/1998Build failed at "Built target kokkoskernel"2017-11-28T20:28:09ZJames WillenbringBuild failed at "Built target kokkoskernel"*Created by: lkkevin*
Hi,
I am trying to build Trilinos 12.12.1 for Peridigm on CentOS 7.4. The "make -j 2" failed at [59] Built target kokkoskernel. I have been working on this issue for a couple day but could not figure out. P...*Created by: lkkevin*
Hi,
I am trying to build Trilinos 12.12.1 for Peridigm on CentOS 7.4. The "make -j 2" failed at [59] Built target kokkoskernel. I have been working on this issue for a couple day but could not figure out. Please help. Thanks.
#------ These are the prerequisites that i installed on my VM: -----#
boost_1_65_1 , cmake-3.10.0-rc5, hdf5-1.10.1, matio-1.5.11, netcdf-c-4.5.0, openmpi-3.0.0, blas 3.8.0, lapack 3.8.0.
#------ This is the CMakeError.log ------#
Performing C++ SOURCE FILE Test HAVE_TEUCHOS_LAPACKLARND failed with the following output:
Change Dir: /root/Downloads/trilinos-12.12.1-Source/build/CMakeFiles/CMakeTmp
Run Build Command:"/usr/bin/gmake" "cmTC_2fa60/fast"
/usr/bin/gmake -f CMakeFiles/cmTC_2fa60.dir/build.make CMakeFiles/cmTC_2fa60.dir/build
gmake[1]: Entering directory `/root/Downloads/trilinos-12.12.1-Source/build/CMakeFiles/CMakeTmp'
Building CXX object CMakeFiles/cmTC_2fa60.dir/src.cxx.o
/usr/local/bin/mpicxx -O2 -std=c++11 -pedantic -ftrapv -Wall -Wno-long-long -std=c++11 -DHAVE_TEUCHOS_LAPACKLARND -O3 -DNDEBUG -o CMakeFiles/cmTC_2fa60.dir/src.cxx.o -c /root/Downloads/trilinos-12.12.1-Source/build/CMakeFiles/CMakeTmp/src.cxx
/root/Downloads/trilinos-12.12.1-Source/build/CMakeFiles/CMakeTmp/src.cxx: In function ‘int main()’:
/root/Downloads/trilinos-12.12.1-Source/build/CMakeFiles/CMakeTmp/src.cxx:12:38: warning: narrowing conversion of ‘0.0’ from ‘double’ to ‘int’ inside { } [-Wnarrowing]
int seed[4] = { 0.0, 0.0, 0.0, 1.0 };
^
/root/Downloads/trilinos-12.12.1-Source/build/CMakeFiles/CMakeTmp/src.cxx:12:38: warning: narrowing conversion of ‘0.0’ from ‘double’ to ‘int’ inside { } [-Wnarrowing]
/root/Downloads/trilinos-12.12.1-Source/build/CMakeFiles/CMakeTmp/src.cxx:12:38: warning: narrowing conversion of ‘0.0’ from ‘double’ to ‘int’ inside { } [-Wnarrowing]
/root/Downloads/trilinos-12.12.1-Source/build/CMakeFiles/CMakeTmp/src.cxx:12:38: warning: narrowing conversion of ‘1.0e+0’ from ‘double’ to ‘int’ inside { } [-Wnarrowing]
Linking CXX executable cmTC_2fa60
/usr/local/bin/cmake -E cmake_link_script CMakeFiles/cmTC_2fa60.dir/link.txt --verbose=1
/usr/local/bin/mpicxx -O2 -std=c++11 -pedantic -ftrapv -Wall -Wno-long-long -std=c++11 -DHAVE_TEUCHOS_LAPACKLARND -O3 -DNDEBUG -rdynamic CMakeFiles/cmTC_2fa60.dir/src.cxx.o -o cmTC_2fa60 /usr/local/lib/liblapack.a
CMakeFiles/cmTC_2fa60.dir/src.cxx.o: In function `main':
src.cxx:(.text.startup+0x22): undefined reference to `dlarnd_'
collect2: error: ld returned 1 exit status
gmake[1]: *** [cmTC_2fa60] Error 1
gmake[1]: Leaving directory `/root/Downloads/trilinos-12.12.1-Source/build/CMakeFiles/CMakeTmp'
gmake: *** [cmTC_2fa60/fast] Error 2
Return value: 1
Source file was:
#define F77_BLAS_MANGLE(name,NAME) name ## _
#define DLARND_F77 F77_BLAS_MANGLE(dlarnd,DLARND)
extern "C" { double DLARND_F77(const int* idist, int* seed); }
int main()
{
const int idist = 1;
int seed[4] = { 0.0, 0.0, 0.0, 1.0 };
double val = DLARND_F77(&idist, seed);
return (val < 0.0 ? 1 : 0);
}
Performing C++ SOURCE FILE Test HAVE_CXX_PRAGMA_WEAK failed with the following output:
Change Dir: /root/Downloads/trilinos-12.12.1-Source/build/CMakeFiles/CMakeTmp
Run Build Command:"/usr/bin/gmake" "cmTC_82e34/fast"
/usr/bin/gmake -f CMakeFiles/cmTC_82e34.dir/build.make CMakeFiles/cmTC_82e34.dir/build
gmake[1]: Entering directory `/root/Downloads/trilinos-12.12.1-Source/build/CMakeFiles/CMakeTmp'
Building CXX object CMakeFiles/cmTC_82e34.dir/src.cxx.o
/usr/local/bin/mpicxx -O2 -std=c++11 -pedantic -ftrapv -Wall -Wno-long-long -std=c++11 -DHAVE_CXX_PRAGMA_WEAK -O3 -DNDEBUG -o CMakeFiles/cmTC_82e34.dir/src.cxx.o -c /root/Downloads/trilinos-12.12.1-Source/build/CMakeFiles/CMakeTmp/src.cxx
Linking CXX executable cmTC_82e34
/usr/local/bin/cmake -E cmake_link_script CMakeFiles/cmTC_82e34.dir/link.txt --verbose=1
/usr/local/bin/mpicxx -O2 -std=c++11 -pedantic -ftrapv -Wall -Wno-long-long -std=c++11 -DHAVE_CXX_PRAGMA_WEAK -O3 -DNDEBUG -rdynamic CMakeFiles/cmTC_82e34.dir/src.cxx.o -o cmTC_82e34
CMakeFiles/cmTC_82e34.dir/src.cxx.o: In function `main':
src.cxx:(.text.startup+0x1c): undefined reference to `A::theFunction()'
collect2: error: ld returned 1 exit status
gmake[1]: *** [cmTC_82e34] Error 1
gmake[1]: Leaving directory `/root/Downloads/trilinos-12.12.1-Source/build/CMakeFiles/CMakeTmp'
gmake: *** [cmTC_82e34/fast] Error 2
Source file was:
#include <iostream>
namespace A {
// theFunction never gets defined, because we
// don't link with a library that defines it.
// That's OK, because it's weak linkage.
#pragma weak theFunction
extern void theFunction ();
}
int main() {
std::cout << "Hi! I am main." << std::endl;
if (A::theFunction != NULL) {
// Should never be called, since we don't link
// with a library that defines A::theFunction.
A::theFunction ();
}
return 0;
}
Determining if the function dggsvd3 exists failed with the following output:
Change Dir: /root/Downloads/trilinos-12.12.1-Source/build/CMakeFiles/CMakeTmp
Run Build Command:"/usr/bin/gmake" "cmTC_427b3/fast"
/usr/bin/gmake -f CMakeFiles/cmTC_427b3.dir/build.make CMakeFiles/cmTC_427b3.dir/build
gmake[1]: Entering directory `/root/Downloads/trilinos-12.12.1-Source/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_427b3.dir/CheckFunctionExists.c.o
/usr/local/bin/mpicc -DCHECK_FUNCTION_EXISTS=dggsvd3 -O3 -DNDEBUG -o CMakeFiles/cmTC_427b3.dir/CheckFunctionExists.c.o -c /usr/local/share/cmake-3.10/Modules/CheckFunctionExists.c
Linking C executable cmTC_427b3
/usr/local/bin/cmake -E cmake_link_script CMakeFiles/cmTC_427b3.dir/link.txt --verbose=1
/usr/local/bin/mpicc -DCHECK_FUNCTION_EXISTS=dggsvd3 -O3 -DNDEBUG -rdynamic CMakeFiles/cmTC_427b3.dir/CheckFunctionExists.c.o -o cmTC_427b3 /usr/local/lib/liblapack.a /usr/local/lib/libblas.a
CMakeFiles/cmTC_427b3.dir/CheckFunctionExists.c.o: In function `main':
CheckFunctionExists.c:(.text.startup+0xc): undefined reference to `dggsvd3'
collect2: error: ld returned 1 exit status
gmake[1]: *** [cmTC_427b3] Error 1
gmake[1]: Leaving directory `/root/Downloads/trilinos-12.12.1-Source/build/CMakeFiles/CMakeTmp'
gmake: *** [cmTC_427b3/fast] Error 2
Determining if the function dggsvd3_ exists failed with the following output:
Change Dir: /root/Downloads/trilinos-12.12.1-Source/build/CMakeFiles/CMakeTmp
Run Build Command:"/usr/bin/gmake" "cmTC_01e16/fast"
/usr/bin/gmake -f CMakeFiles/cmTC_01e16.dir/build.make CMakeFiles/cmTC_01e16.dir/build
gmake[1]: Entering directory `/root/Downloads/trilinos-12.12.1-Source/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_01e16.dir/CheckFunctionExists.c.o
/usr/local/bin/mpicc -DCHECK_FUNCTION_EXISTS=dggsvd3_ -O3 -DNDEBUG -o CMakeFiles/cmTC_01e16.dir/CheckFunctionExists.c.o -c /usr/local/share/cmake-3.10/Modules/CheckFunctionExists.c
Linking C executable cmTC_01e16
/usr/local/bin/cmake -E cmake_link_script CMakeFiles/cmTC_01e16.dir/link.txt --verbose=1
/usr/local/bin/mpicc -DCHECK_FUNCTION_EXISTS=dggsvd3_ -O3 -DNDEBUG -rdynamic CMakeFiles/cmTC_01e16.dir/CheckFunctionExists.c.o -o cmTC_01e16 /usr/local/lib/liblapack.a /usr/local/lib/libblas.a
/usr/bin/ld: /usr/local/lib/liblapack.a(dlaqps.o): undefined reference to symbol 'lround@@GLIBC_2.2.5'
/usr/lib64/libm.so.6: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
gmake[1]: *** [cmTC_01e16] Error 1
gmake[1]: Leaving directory `/root/Downloads/trilinos-12.12.1-Source/build/CMakeFiles/CMakeTmp'
gmake: *** [cmTC_01e16/fast] Error 2
Determining if the function DGGSVD3 exists failed with the following output:
Change Dir: /root/Downloads/trilinos-12.12.1-Source/build/CMakeFiles/CMakeTmp
Run Build Command:"/usr/bin/gmake" "cmTC_e1f49/fast"
/usr/bin/gmake -f CMakeFiles/cmTC_e1f49.dir/build.make CMakeFiles/cmTC_e1f49.dir/build
gmake[1]: Entering directory `/root/Downloads/trilinos-12.12.1-Source/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_e1f49.dir/CheckFunctionExists.c.o
/usr/local/bin/mpicc -DCHECK_FUNCTION_EXISTS=DGGSVD3 -O3 -DNDEBUG -o CMakeFiles/cmTC_e1f49.dir/CheckFunctionExists.c.o -c /usr/local/share/cmake-3.10/Modules/CheckFunctionExists.c
Linking C executable cmTC_e1f49
/usr/local/bin/cmake -E cmake_link_script CMakeFiles/cmTC_e1f49.dir/link.txt --verbose=1
/usr/local/bin/mpicc -DCHECK_FUNCTION_EXISTS=DGGSVD3 -O3 -DNDEBUG -rdynamic CMakeFiles/cmTC_e1f49.dir/CheckFunctionExists.c.o -o cmTC_e1f49 /usr/local/lib/liblapack.a /usr/local/lib/libblas.a
CMakeFiles/cmTC_e1f49.dir/CheckFunctionExists.c.o: In function `main':
CheckFunctionExists.c:(.text.startup+0xc): undefined reference to `DGGSVD3'
collect2: error: ld returned 1 exit status
gmake[1]: *** [cmTC_e1f49] Error 1
gmake[1]: Leaving directory `/root/Downloads/trilinos-12.12.1-Source/build/CMakeFiles/CMakeTmp'
gmake: *** [cmTC_e1f49/fast] Error 2
Determining if the function DGGSVD3_ exists failed with the following output:
Change Dir: /root/Downloads/trilinos-12.12.1-Source/build/CMakeFiles/CMakeTmp
Run Build Command:"/usr/bin/gmake" "cmTC_6d0e5/fast"
/usr/bin/gmake -f CMakeFiles/cmTC_6d0e5.dir/build.make CMakeFiles/cmTC_6d0e5.dir/build
gmake[1]: Entering directory `/root/Downloads/trilinos-12.12.1-Source/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_6d0e5.dir/CheckFunctionExists.c.o
/usr/local/bin/mpicc -DCHECK_FUNCTION_EXISTS=DGGSVD3_ -O3 -DNDEBUG -o CMakeFiles/cmTC_6d0e5.dir/CheckFunctionExists.c.o -c /usr/local/share/cmake-3.10/Modules/CheckFunctionExists.c
Linking C executable cmTC_6d0e5
/usr/local/bin/cmake -E cmake_link_script CMakeFiles/cmTC_6d0e5.dir/link.txt --verbose=1
/usr/local/bin/mpicc -DCHECK_FUNCTION_EXISTS=DGGSVD3_ -O3 -DNDEBUG -rdynamic CMakeFiles/cmTC_6d0e5.dir/CheckFunctionExists.c.o -o cmTC_6d0e5 /usr/local/lib/liblapack.a /usr/local/lib/libblas.a
CMakeFiles/cmTC_6d0e5.dir/CheckFunctionExists.c.o: In function `main':
CheckFunctionExists.c:(.text.startup+0xc): undefined reference to `DGGSVD3_'
collect2: error: ld returned 1 exit status
gmake[1]: *** [cmTC_6d0e5] Error 1
gmake[1]: Leaving directory `/root/Downloads/trilinos-12.12.1-Source/build/CMakeFiles/CMakeTmp'
gmake: *** [cmTC_6d0e5/fast] Error 2
#------ This is my build.sh -----#
#!/bin/bash
#===========================================================
make clean
rm -rf CMake*
cmake \
-D CMAKE_INSTALL_PREFIX:PATH=/usr/local/trilinos \
-D MPI_BASE_DIR:PATH=/usr/local/lib/openmpi \
-D CMAKE_CXX_FLAGS:STRING="-O2 -std=c++11 -pedantic -ftrapv -Wall -Wno-long-long" \
-D CMAKE_BUILD_TYPE:STRING=RELEASE \
-D Trilinos_WARNINGS_AS_ERRORS_FLAGS:STRING="" \
-D Trilinos_ENABLE_ALL_PACKAGES:BOOL=OFF \
-D Trilinos_ENABLE_Teuchos:BOOL=ON \
-D Trilinos_ENABLE_Shards:BOOL=ON \
-D Trilinos_ENABLE_Sacado:BOOL=ON \
-D Trilinos_ENABLE_Epetra:BOOL=ON \
-D Trilinos_ENABLE_EpetraExt:BOOL=ON \
-D Trilinos_ENABLE_Ifpack:BOOL=ON \
-D Trilinos_ENABLE_AztecOO:BOOL=ON \
-D Trilinos_ENABLE_Amesos:BOOL=ON \
-D Trilinos_ENABLE_Anasazi:BOOL=ON \
-D Trilinos_ENABLE_Belos:BOOL=ON \
-D Trilinos_ENABLE_ML:BOOL=ON \
-D Trilinos_ENABLE_Phalanx:BOOL=ON \
-D Trilinos_ENABLE_Intrepid:BOOL=ON \
-D Trilinos_ENABLE_NOX:BOOL=ON \
-D Trilinos_ENABLE_Stratimikos:BOOL=ON \
-D Trilinos_ENABLE_Thyra:BOOL=ON \
-D Trilinos_ENABLE_Rythmos:BOOL=ON \
-D Trilinos_ENABLE_MOOCHO:BOOL=ON \
-D Trilinos_ENABLE_TriKota:BOOL=OFF \
-D Trilinos_ENABLE_Stokhos:BOOL=ON \
-D Trilinos_ENABLE_Zoltan:BOOL=ON \
-D Trilinos_ENABLE_Piro:BOOL=ON \
-D Trilinos_ENABLE_Teko:BOOL=ON \
-D Trilinos_ENABLE_SEACASIoss:BOOL=ON \
-D Trilinos_ENABLE_SEACAS:BOOL=ON \
-D Trilinos_ENABLE_SEACASBlot:BOOL=ON \
-D Trilinos_ENABLE_Pamgen:BOOL=ON \
-D Trilinos_ENABLE_EXAMPLES:BOOL=OFF \
-D Trilinos_ENABLE_TESTS:BOOL=ON \
-D TPL_ENABLE_HDF5:BOOL=ON \
-D HDF5_INCLUDE_DIRS:PATH=/usr/local/HDF5/include \
-D HDF5_LIBRARY_DIRS:PATH=/usr/local/HDF5/lib \
-D TPL_ENABLE_Netcdf:BOOL=ON \
-D Netcdf_INCLUDE_DIRS:PATH=/usr/local/netcdf/include \
-D Netcdf_LIBRARY_DIRS:PATH=/usr/local/netcdf/lib \
-D TPL_ENABLE_MPI:BOOL=ON \
-D TPL_ENABLE_BLAS:BOOL=ON \
-D TPL_ENABLE_LAPACK:BOOL=ON \
-D TPL_ENABLE_Boost:BOOL=ON \
-D Boost_INCLUDE_DIRS:PATH=/usr/local/boost/include \
-D Boost_LIBRARY_DIRS:PATH=/usr/local/boost/lib \
-D BLAS_LIBRARY_NAMES:STRING=libblas.a \
-D BLAS_LIBRARY_DIRS:STRING=/usr/local/lib \
-D LAPACK_LIBRARY_NAMES:STRING=liblapack.a \
-D LAPACK_LIBRARY_DIRS:STRING=/usr/local/lib \
-D CMAKE_VERBOSE_MAKEFILE:BOOL=OFF \
-D Trilinos_VERBOSE_CONFIGURE:BOOL=OFF \
../
#---------------------------------------------------------------------#
<!---
Note that anything between these delimiters is a comment that will not appear
in the issue description once created. Click on the Preview tab to see what
everything will look like when you submit.
-->
<!---
Feel free to delete anything from this template that is not applicable to the
issue you are submitting.
-->
<!---
Replace <teamName> below with the appropriate Trilinos package/team name.
-->
@trilinos/<teamName>
<!---
Assignees: If you know anyone who should likely tackle this issue, select them
from the Assignees drop-down on the right.
-->
<!---
Lables: Choose any applicable package names from the Labels drop-down on the
right. Additionally, choose a label to indicate the type of issue, for
instance, bug, build, documentation, enhancement, etc.
-->
## Expectations
<!---
Tell us what you think should happen, how you think things should work, what
you would like to see in the documentation, etc.
-->
## Current Behavior
<!---
Tell us how the current behavior fails to meet your expectations in some way.
-->
## Motivation and Context
<!---
How has this expectation failure affected you? What are you trying to
accomplish? Why do we need to address this? What does it have to do with
anything? Providing context helps us come up with a solution that is most
useful in the real world.
-->
## Definition of Done
<!---
Tell us what needs to happen. If necessary, give us a task lisk along the
lines of:
- [ ] First do this.
- [ ] Then do that.
- [ ] Also this other thing.
-->
## Possible Solution
<!---
Not obligatory, but suggest a fix for the bug or documentation, or suggest
ideas on how to implement the addition or change.
-->
## Steps to Reproduce
<!---
Provide a link to a live example, or an unambiguous set of steps to reproduce
this issue. Include code to reproduce, if relevant.
1. Do this.
1. Do that.
1. Shake fist angrily at computer.
-->
## Your Environment
<!---
Include relevant details about your environment such that we can replicate this
issue.
-->
- **Relevant repo SHA1s:**
- **Relevant configure flags or configure script:**
- **Operating system and version:**
- **Compiler and TPL versions:**
## Related Issues
<!---
If applicable, let us know how this bug is related to any other open issues:
-->
* Blocks
* Is blocked by
* Follows
* Precedes
* Related to
* Part of
* Composed of
## Additional Information
<!---
Anything else that might be helpful for us to know in addressing this issue:
* Configure log file:
* Build log file:
* Test log file:
* When was the last time everything worked (date/time; SHA1s; etc.)?
* What did you do that made the bug rear its ugly head?
* Have you tried turning it off and on again?
-->
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/2057TpetraExt MMDetails / MM multiply compile error2017-12-06T20:40:33ZJames WillenbringTpetraExt MMDetails / MM multiply compile error*Created by: jjellio*
<!--- Provide a general summary of the issue in the Title above. -->
I've pulled develop, and when I try building on Cori with Intel 17, I am seeing the following error related to TpetraExt not finding the new mat...*Created by: jjellio*
<!--- Provide a general summary of the issue in the Title above. -->
I've pulled develop, and when I try building on Cori with Intel 17, I am seeing the following error related to TpetraExt not finding the new matrix matrix multiply routines.
## Compile error:
```
/global/homes/j/jjellio/src/Trilinos-dev/packages/tpetra/core/ext/TpetraExt_MatrixMatrix_def.hpp(3366): error: class "Tpetra::MMdetails::KernelWrappers<Tpetra::Details::DefaultTypes::scalar_type={double}, Tpetra::Details::DefaultTypes::local_ordinal_type={int}, Tpetra::Details::DefaultTypes::global_ordinal_type={int}, Tpetra::Details::DefaultTypes::node_type>" has no member "jacobi_A_B_newmatrix_kernel_wrapper"
KernelWrappers<Scalar,LocalOrdinal,GlobalOrdinal,Node>::jacobi_A_B_newmatrix_kernel_wrapper(omega,Dinv,Aview,Bview,targetMapToOrigRow,targetMapToImportRow,Bcol2Ccol,Icol2Ccol,C,Cimport,label,params);
^
detected during:
instantiation of "void Tpetra::MMdetails::jacobi_A_B_newmatrix(Scalar, const Tpetra::Vector<Scalar, LocalOrdinal, GlobalOrdinal, Node, Node::classic> &, Tpetra::CrsMatrixStruct<Scalar, LocalOrdinal, GlobalOrdinal, Node> &, Tpetra::CrsMatrixStruct<Scalar, LocalOrdinal, GlobalOrdinal, Node> &, Tpetra::CrsMatrix<Scalar, LocalOrdinal, GlobalOrdinal, Node, Node::classic> &, const std::string &, const Teuchos::RCP<Teuchos::ParameterList> &) [with Scalar=double,
LocalOrdinal=Tpetra::Details::DefaultTypes::local_ordinal_type={int}, GlobalOrdinal=Tpetra::Details::DefaultTypes::global_ordinal_type={int}, Node=Tpetra::Details::DefaultTypes::node_type]" at line 386
instantiation of "void Tpetra::MatrixMatrix::Jacobi(Scalar, const Tpetra::Vector<Scalar, LocalOrdinal, GlobalOrdinal, Node, Node::classic> &, const Tpetra::CrsMatrix<Scalar, LocalOrdinal, GlobalOrdinal, Node, Node::classic> &, const Tpetra::CrsMatrix<Scalar, LocalOrdinal, GlobalOrdinal, Node, Node::classic> &, Tpetra::CrsMatrix<Scalar, LocalOrdinal, GlobalOrdinal, Node, Node::classic> &, bool, const std::string &, const Teuchos::RCP<Teuchos::ParameterList> &) [with Scalar=double,
LocalOrdinal=Tpetra::Details::DefaultTypes::local_ordinal_type={int}, GlobalOrdinal=Tpetra::Details::DefaultTypes::global_ordinal_type={int}, Node=Tpetra::Details::DefaultTypes::node_type]" at line 55 of "/global/homes/j/jjellio/src/Trilinos-dev/packages/tpetra/core/ext/TpetraExt_MatrixMatrix.cpp"
/global/homes/j/jjellio/src/Trilinos-dev/packages/tpetra/core/ext/TpetraExt_MatrixMatrix_def.hpp(3366): error: class "Tpetra::MMdetails::KernelWrappers<double, int, Tpetra::longlong={long long}, Tpetra::Kokkos_Compat_KokkosOpenMPWrapperNode>" has no member "jacobi_A_B_newmatrix_kernel_wrapper"
KernelWrappers<Scalar,LocalOrdinal,GlobalOrdinal,Node>::jacobi_A_B_newmatrix_kernel_wrapper(omega,Dinv,Aview,Bview,targetMapToOrigRow,targetMapToImportRow,Bcol2Ccol,Icol2Ccol,C,Cimport,label,params);
^
detected during:
instantiation of "void Tpetra::MMdetails::jacobi_A_B_newmatrix(Scalar, const Tpetra::Vector<Scalar, LocalOrdinal, GlobalOrdinal, Node, Node::classic> &, Tpetra::CrsMatrixStruct<Scalar, LocalOrdinal, GlobalOrdinal, Node> &, Tpetra::CrsMatrixStruct<Scalar, LocalOrdinal, GlobalOrdinal, Node> &, Tpetra::CrsMatrix<Scalar, LocalOrdinal, GlobalOrdinal, Node, Node::classic> &, const std::string &, const Teuchos::RCP<Teuchos::ParameterList> &) [with Scalar=double, LocalOrdinal=int,
GlobalOrdinal=Tpetra::longlong={long long}, Node=Tpetra::Kokkos_Compat_KokkosOpenMPWrapperNode]" at line 386
instantiation of "void Tpetra::MatrixMatrix::Jacobi(Scalar, const Tpetra::Vector<Scalar, LocalOrdinal, GlobalOrdinal, Node, Node::classic> &, const Tpetra::CrsMatrix<Scalar, LocalOrdinal, GlobalOrdinal, Node, Node::classic> &, const Tpetra::CrsMatrix<Scalar, LocalOrdinal, GlobalOrdinal, Node, Node::classic> &, Tpetra::CrsMatrix<Scalar, LocalOrdinal, GlobalOrdinal, Node, Node::classic> &, bool, const std::string &, const Teuchos::RCP<Teuchos::ParameterList> &) [with Scalar=double,
LocalOrdinal=int, GlobalOrdinal=Tpetra::longlong={long long}, Node=Tpetra::Kokkos_Compat_KokkosOpenMPWrapperNode]" at line 55 of "/global/homes/j/jjellio/src/Trilinos-dev/packages/tpetra/core/ext/TpetraExt_MatrixMatrix.cpp"
/global/homes/j/jjellio/src/Trilinos-dev/packages/tpetra/core/ext/TpetraExt_MatrixMatrix_def.hpp(3366): error: class "Tpetra::MMdetails::KernelWrappers<int, int, int, Tpetra::Kokkos_Compat_KokkosOpenMPWrapperNode>" has no member "jacobi_A_B_newmatrix_kernel_wrapper"
KernelWrappers<Scalar,LocalOrdinal,GlobalOrdinal,Node>::jacobi_A_B_newmatrix_kernel_wrapper(omega,Dinv,Aview,Bview,targetMapToOrigRow,targetMapToImportRow,Bcol2Ccol,Icol2Ccol,C,Cimport,label,params);
^
detected during:
instantiation of "void Tpetra::MMdetails::jacobi_A_B_newmatrix(Scalar, const Tpetra::Vector<Scalar, LocalOrdinal, GlobalOrdinal, Node, Node::classic> &, Tpetra::CrsMatrixStruct<Scalar, LocalOrdinal, GlobalOrdinal, Node> &, Tpetra::CrsMatrixStruct<Scalar, LocalOrdinal, GlobalOrdinal, Node> &, Tpetra::CrsMatrix<Scalar, LocalOrdinal, GlobalOrdinal, Node, Node::classic> &, const std::string &, const Teuchos::RCP<Teuchos::ParameterList> &) [with Scalar=int, LocalOrdinal=int,
GlobalOrdinal=int, Node=Tpetra::Kokkos_Compat_KokkosOpenMPWrapperNode]" at line 386
instantiation of "void Tpetra::MatrixMatrix::Jacobi(Scalar, const Tpetra::Vector<Scalar, LocalOrdinal, GlobalOrdinal, Node, Node::classic> &, const Tpetra::CrsMatrix<Scalar, LocalOrdinal, GlobalOrdinal, Node, Node::classic> &, const Tpetra::CrsMatrix<Scalar, LocalOrdinal, GlobalOrdinal, Node, Node::classic> &, Tpetra::CrsMatrix<Scalar, LocalOrdinal, GlobalOrdinal, Node, Node::classic> &, bool, const std::string &, const Teuchos::RCP<Teuchos::ParameterList> &) [with Scalar=int,
LocalOrdinal=int, GlobalOrdinal=int, Node=Tpetra::Kokkos_Compat_KokkosOpenMPWrapperNode]" at line 55 of "/global/homes/j/jjellio/src/Trilinos-dev/packages/tpetra/core/ext/TpetraExt_MatrixMatrix.cpp"
/global/homes/j/jjellio/src/Trilinos-dev/packages/tpetra/core/ext/TpetraExt_MatrixMatrix_def.hpp(3366): error: class "Tpetra::MMdetails::KernelWrappers<Tpetra::longlong={long long}, int, Tpetra::longlong={long long}, Tpetra::Kokkos_Compat_KokkosOpenMPWrapperNode>" has no member "jacobi_A_B_newmatrix_kernel_wrapper"
KernelWrappers<Scalar,LocalOrdinal,GlobalOrdinal,Node>::jacobi_A_B_newmatrix_kernel_wrapper(omega,Dinv,Aview,Bview,targetMapToOrigRow,targetMapToImportRow,Bcol2Ccol,Icol2Ccol,C,Cimport,label,params);
^
detected during:
instantiation of "void Tpetra::MMdetails::jacobi_A_B_newmatrix(Scalar, const Tpetra::Vector<Scalar, LocalOrdinal, GlobalOrdinal, Node, Node::classic> &, Tpetra::CrsMatrixStruct<Scalar, LocalOrdinal, GlobalOrdinal, Node> &, Tpetra::CrsMatrixStruct<Scalar, LocalOrdinal, GlobalOrdinal, Node> &, Tpetra::CrsMatrix<Scalar, LocalOrdinal, GlobalOrdinal, Node, Node::classic> &, const std::string &, const Teuchos::RCP<Teuchos::ParameterList> &) [with Scalar=Tpetra::longlong={long long},
LocalOrdinal=int, GlobalOrdinal=Tpetra::longlong={long long}, Node=Tpetra::Kokkos_Compat_KokkosOpenMPWrapperNode]" at line 386
instantiation of "void Tpetra::MatrixMatrix::Jacobi(Scalar, const Tpetra::Vector<Scalar, LocalOrdinal, GlobalOrdinal, Node, Node::classic> &, const Tpetra::CrsMatrix<Scalar, LocalOrdinal, GlobalOrdinal, Node, Node::classic> &, const Tpetra::CrsMatrix<Scalar, LocalOrdinal, GlobalOrdinal, Node, Node::classic> &, Tpetra::CrsMatrix<Scalar, LocalOrdinal, GlobalOrdinal, Node, Node::classic> &, bool, const std::string &, const Teuchos::RCP<Teuchos::ParameterList> &) [with
Scalar=Tpetra::longlong={long long}, LocalOrdinal=int, GlobalOrdinal=Tpetra::longlong={long long}, Node=Tpetra::Kokkos_Compat_KokkosOpenMPWrapperNode]" at line 55 of "/global/homes/j/jjellio/src/Trilinos-dev/packages/tpetra/core/ext/TpetraExt_MatrixMatrix.cpp"
/global/homes/j/jjellio/src/Trilinos-dev/packages/tpetra/core/ext/TpetraExt_MatrixMatrix_def.hpp(3366): error: class "Tpetra::MMdetails::KernelWrappers<int, int, Tpetra::longlong={long long}, Tpetra::Kokkos_Compat_KokkosOpenMPWrapperNode>" has no member "jacobi_A_B_newmatrix_kernel_wrapper"
KernelWrappers<Scalar,LocalOrdinal,GlobalOrdinal,Node>::jacobi_A_B_newmatrix_kernel_wrapper(omega,Dinv,Aview,Bview,targetMapToOrigRow,targetMapToImportRow,Bcol2Ccol,Icol2Ccol,C,Cimport,label,params);
^
detected during:
instantiation of "void Tpetra::MMdetails::jacobi_A_B_newmatrix(Scalar, const Tpetra::Vector<Scalar, LocalOrdinal, GlobalOrdinal, Node, Node::classic> &, Tpetra::CrsMatrixStruct<Scalar, LocalOrdinal, GlobalOrdinal, Node> &, Tpetra::CrsMatrixStruct<Scalar, LocalOrdinal, GlobalOrdinal, Node> &, Tpetra::CrsMatrix<Scalar, LocalOrdinal, GlobalOrdinal, Node, Node::classic> &, const std::string &, const Teuchos::RCP<Teuchos::ParameterList> &) [with Scalar=int, LocalOrdinal=int,
GlobalOrdinal=Tpetra::longlong={long long}, Node=Tpetra::Kokkos_Compat_KokkosOpenMPWrapperNode]" at line 386
instantiation of "void Tpetra::MatrixMatrix::Jacobi(Scalar, const Tpetra::Vector<Scalar, LocalOrdinal, GlobalOrdinal, Node, Node::classic> &, const Tpetra::CrsMatrix<Scalar, LocalOrdinal, GlobalOrdinal, Node, Node::classic> &, const Tpetra::CrsMatrix<Scalar, LocalOrdinal, GlobalOrdinal, Node, Node::classic> &, Tpetra::CrsMatrix<Scalar, LocalOrdinal, GlobalOrdinal, Node, Node::classic> &, bool, const std::string &, const Teuchos::RCP<Teuchos::ParameterList> &) [with Scalar=int,
LocalOrdinal=int, GlobalOrdinal=Tpetra::longlong={long long}, Node=Tpetra::Kokkos_Compat_KokkosOpenMPWrapperNode]" at line 76 of "/global/homes/j/jjellio/src/Trilinos-dev/packages/tpetra/core/ext/TpetraExt_MatrixMatrix.cpp"
compilation aborted for /global/homes/j/jjellio/src/Trilinos-dev/packages/tpetra/core/ext/TpetraExt_MatrixMatrix.cpp (code 2)
packages/tpetra/core/ext/CMakeFiles/tpetraext.dir/build.make:62: recipe for target 'packages/tpetra/core/ext/CMakeFiles/tpetraext.dir/TpetraExt_MatrixMatrix.cpp.o' failed
make[2]: *** [packages/tpetra/core/ext/CMakeFiles/tpetraext.dir/TpetraExt_MatrixMatrix.cpp.o] Error 2
make[2]: *** Waiting for unfinished jobs....
CMakeFiles/Makefile2:5417: recipe for target 'packages/tpetra/core/ext/CMakeFiles/tpetraext.dir/all' failed
make[1]: *** [packages/tpetra/core/ext/CMakeFiles/tpetraext.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
```
## CMake:
```
cmake -D Trilinos_LINK_SEARCH_START_STATIC:BOOL=ON
-D MPI_USE_COMPILER_WRAPPERS=OFF
-D CMAKE_CXX_COMPILER="CC"
-D CMAKE_C_COMPILER="cc"
-D CMAKE_CXX_FLAGS="-xMIC-AVX512 -mkl=parallel -g1"
-D CMAKE_C_FLAGS="-xMIC-AVX512 -mkl=parallel -g1"
-D CMAKE_Fortran_FLAGS="-xMIC-AVX512 -mkl=parallel -g1"
-D CMAKE_EXE_LINKER_FLAGS="-xMIC-AVX512 -mkl=parallel "
-D CMAKE_Fortran_COMPILER="ftn"
-D HAVE_GCC_ABI_DEMANGLE:BOOL=TRUE
-D Trilinos_ENABLE_EXPLICIT_INSTANTIATION:BOOL=ON
-D Trilinos_ENABLE_DEBUG:BOOL=OFF
-D CMAKE_INSTALL_PREFIX:PATH="/global/homes/j/jjellio/install/trilinos-12.13/intel-17.0.2.174_mpt-7.6.0_knl-openmp_hp-2-dev"
-D Trilinos_ENABLE_INSTALL_CMAKE_CONFIG_FILES:BOOL=ON
-D CMAKE_BUILD_TYPE:STRING=RELEASE
-D CMAKE_VERBOSE_MAKEFILE:BOOL=OFF
-D Trilinos_ENABLE_ALL_PACKAGES:BOOL=OFF
-D Trilinos_ENABLE_ALL_OPTIONAL_PACKAGES:BOOL=ON
-D Trilinos_ENABLE_TESTS:BOOL=OFF
-D Trilinos_ENABLE_EXAMPLES:BOOL=OFF
-D BUILD_SHARED_LIBS:BOOL=OFF
-D Trilinos_SET_INSTALL_RPATH=OFF
-D DART_TESTING_TIMEOUT:STRING=200
-D Trilinos_ENABLE_CXX11=ON
-D TPL_ENABLE_MPI=ON
-D MPI_EXEC:PATH="srun"
-D MPI_EXEC_NUMPROCS_FLAG:STRING="-c;8;--cpu_bind=cores;-n"
-D TPL_ENABLE_BLAS=ON
-D BLAS_LIBRARY_DIRS:PATH="/opt/intel/compilers_and_libraries_2017.2.174/linux/mkl/lib/intel64;/opt/intel/compilers_and_libraries_2017.2.174/linux/mkl/../compiler/lib/intel64_lin/;/opt/intel/compilers_and_libraries_2017.2.174/linux/lib/intel64_lin/"
-D BLAS_LIBRARY_NAMES:STRING="mkl_intel_lp64;mkl_intel_thread;mkl_core;iomp5;m;dl"
-D TPL_ENABLE_LAPACK=ON
-D LAPACK_LIBRARY_DIRS:PATH="/opt/intel/compilers_and_libraries_2017.2.174/linux/mkl/lib/intel64;/opt/intel/compilers_and_libraries_2017.2.174/linux/mkl/../compiler/lib/intel64_lin/;/opt/intel/compilers_and_libraries_2017.2.174/linux/lib/intel64_lin/"
-D LAPACK_LIBRARY_NAMES:STRING="mkl_intel_lp64;mkl_intel_thread;mkl_core;iomp5;m;dl"
-D TPL_ENABLE_Zlib=ON
-D TPL_ENABLE_DLlib=ON
-D TPL_ENABLE_yaml-cpp=ON
-D yaml-cpp_LIBRARY_DIRS:PATH=/global/homes/j/jjellio/install/yaml-cpp/intel-17.0.1.132_mpt-7.4.4_hsw/yaml-cpp/lib
-D yaml-cpp_INCLUDE_DIRS:PATH=/global/homes/j/jjellio/install/yaml-cpp/intel-17.0.1.132_mpt-7.4.4_hsw/yaml-cpp/include
-D TPL_ENABLE_Boost=ON
-D Boost_INCLUDE_DIRS:PATH="/usr/common/software/boost/1.62/intel/mic-knl/include"
-D Boost_LIBRARY_DIRS:PATH="/usr/common/software/boost/1.62/intel/mic-knl/lib"
-D TPL_ENABLE_BoostLib=ON
-D BoostLib_INCLUDE_DIRS:PATH="/usr/common/software/boost/1.62/intel/mic-knl/include"
-D BoostLib_LIBRARY_DIRS:PATH="/usr/common/software/boost/1.62/intel/mic-knl/lib"
-D TPL_ENABLE_Netcdf=ON
-D Netcdf_INCLUDE_DIRS:PATH="/opt/cray/pe/parallel-netcdf/1.8.0/intel/15.0/include;/global/homes/j/jjellio/install/netcdf-4.1.1.1/include"
-D Netcdf_LIBRARY_DIRS:PATH="/opt/cray/pe/parallel-netcdf/1.8.0/intel/15.0/lib;/global/homes/j/jjellio/install/netcdf-4.1.1.1/lib;/opt/cray/pe/hdf5-parallel/1.10.0.1/INTEL/15.0/lib"
-D Netcdf_LIBRARY_NAMES:STRING="netcdf;pnetcdf;hdf5hl_fortran;hdf5_fortran;hdf5_hl;hdf5;z"
-D TPL_Netcdf_Enables_Netcdf4:BOOL=ON
-D TPL_Netcdf_Enables_PNetcdf:BOOL=ON
-D TPL_Netcdf_PARALLEL:BOOL=ON
-D TPL_ENABLE_HDF5=ON
-D HDF5_INCLUDE_DIRS:PATH="/opt/cray/pe/hdf5-parallel/1.10.0.1/INTEL/15.0/include"
-D HDF5_LIBRARY_DIRS:PATH="/opt/cray/pe/hdf5-parallel/1.10.0.1/INTEL/15.0/lib"
-D HDF5_LIBRARY_NAMES:STRING="hdf5hl_fortran;hdf5_fortran;hdf5_hl;hdf5;z"
-D TPL_ENABLE_METIS=OFF
-D METIS_INCLUDE_DIRS:PATH="/usr/common/software/metis/5.1.0/include"
-D METIS_LIBRARY_DIRS:PATH="/usr/common/software/metis/5.1.0/lib"
-D TPL_ENABLE_ParMETIS=OFF
-D ParMETIS_INCLUDE_DIRS:PATH="/global/homes/j/jjellio/install/parmetis-4.0.3/include;/usr/common/software/metis/5.1.0/include"
-D ParMETIS_LIBRARY_DIRS:PATH="/global/homes/j/jjellio/install/parmetis-4.0.3/lib;/usr/common/software/metis/5.1.0/lib"
-D ParMETIS_LIBRARY_NAMES:STRING="parmetis;metis"
-D Trilinos_ENABLE_Kokkos=ON
-D Kokkos_ENABLE_TESTS:BOOL=ON
-D Trilinos_ENABLE_OpenMP=ON
-D Kokkos_ENABLE_Serial:BOOL=ON
-D Kokkos_ENABLE_OpenMP:BOOL=ON
-D Kokkos_ENABLE_Pthread:BOOL=OFF
-D Kokkos_ENABLE_Cuda:BOOL=OFF
-D TPL_ENABLE_Pthread=OFF
-D TPL_ENABLE_CUDA=OFF
-D Kokkos_ENABLE_Cuda_UVM:BOOL=ON
-D Teuchos_ENABLE_COMPLEX=OFF
-D Trilinos_ENABLE_Tpetra=ON
-D Tpetra_INST_SERIAL:BOOL=ON
-D Tpetra_INST_OPENMP:BOOL=ON
-D Tpetra_INST_PTHREAD:BOOL=OFF
-D Tpetra_INST_CUDA:BOOL=OFF
-D Tpetra_INST_COMPLEX_DOUBLE:BOOL=OFF
-D Tpetra_ENABLE_TESTS:BOOL=OFF
-D Tpetra_ENABLE_EXAMPLES:BOOL=OFF
-D Trilinos_ENABLE_Belos=ON
-D Tpetra_ENABLE_MMM_Timings:BOOL=ON
-D Trilinos_ENABLE_MueLu=ON
-D MueLu_ENABLE_TESTS=OFF
-D MueLu_ENABLE_EXAMPLES=OFF
-D KokkosKernels_ENABLE_Experimental:BOOL=ON
-D TpetraKernels_ENABLE_Experimental:BOOL=ON
-D MueLu_ENABLE_Experimental:BOOL=ON
-D MueLu_ENABLE_Kokkos_Refactor:BOOL=ON
-D Xpetra_ENABLE_Experimental:BOOL=ON
-D Xpetra_ENABLE_Kokkos_Refactor:BOOL=ON
-D Trilinos_ENABLE_Xpetra=ON
-D Trilinos_ENABLE_Isorropia=ON
-D Trilinos_ENABLE_Galeri=ON
-D Trilinos_ENABLE_Teuchos=ON
-D Trilinos_ENABLE_Epetra=ON
-D Trilinos_ENABLE_EpetraExt=ON
-D Trilinos_ENABLE_Anasazi=ON
-D Anasazi_ENABLE_RBGen=ON
-D Trilinos_ENABLE_Stokhos=OFF
-D Trilinos_ENABLE_Amesos2=ON
-D Amesos2_ENABLE_KLU2=ON
-D Trilinos_ENABLE_Ifpack2=ON
-D Trilinos_ENABLE_Zoltan2=ON
-D Trilinos_ENABLE_Panzer=ON
-D Panzer_ENABLE_TESTS=ON
-D Panzer_ENABLE_EXAMPLES=ON
/global/homes/j/jjellio/src/Trilinos-dev
```
<!---
Replace <teamName> below with the appropriate Trilinos package/team name.
-->
@trilinos/tpetra
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/1818kokkos-kernels vs kokkoskernels2017-11-01T15:34:48ZJames Willenbringkokkos-kernels vs kokkoskernels*Created by: nschloe*
There is a discrepancy in files names for the new Kokkos kernels:
* The package directory is called `kokkos-kernels`.
* The library file name is `kokkoskernels.so`.
This creates some problems on the Debi...*Created by: nschloe*
There is a discrepancy in files names for the new Kokkos kernels:
* The package directory is called `kokkos-kernels`.
* The library file name is `kokkoskernels.so`.
This creates some problems on the Debian side. I'd be great if this could be made consistent one way or the other. For comparison, check `trilinoscouplings`.https://gitlab.osti.gov/jmwille/Trilinos/-/issues/1729kokkos-kernels: Turn off unneeded instantiations by default2017-10-26T21:10:54ZJames Willenbringkokkos-kernels: Turn off unneeded instantiations by default*Created by: mhoemmen*
@trilinos/tpetra
Trilinos doesn't need the following instantiations:
- `CudaSpace` (not used, Trilinos assumes UVM and uses `CudaUVMSpace`)
- Unused offset types
See `${PACKAGE_NAME}_INST_MEMSPACE_C...*Created by: mhoemmen*
@trilinos/tpetra
Trilinos doesn't need the following instantiations:
- `CudaSpace` (not used, Trilinos assumes UVM and uses `CudaUVMSpace`)
- Unused offset types
See `${PACKAGE_NAME}_INST_MEMSPACE_CUDASPACE` in `kokkos-kernels/CMakeLists.txt`. I'm not sure if Tpetra can control these directly; it may be necessary to change defaults in kokkos-kernels.https://gitlab.osti.gov/jmwille/Trilinos/-/issues/1622KokkosKernels: Gauss-Seidel threaded setup performance issues with Ifpack2 an...2017-09-12T19:15:52ZJames WillenbringKokkosKernels: Gauss-Seidel threaded setup performance issues with Ifpack2 and MueLu*Created by: pwxy*
I am trying to use the KokkosKernels threaded Gauss-Seidel.
I'm calling it through ifpack2 ("MT Gauss-Seidel") as a smoother for MueLu,
so the problem could be a bad interaction between KokkosKernels and ifpack2 or ...*Created by: pwxy*
I am trying to use the KokkosKernels threaded Gauss-Seidel.
I'm calling it through ifpack2 ("MT Gauss-Seidel") as a smoother for MueLu,
so the problem could be a bad interaction between KokkosKernels and ifpack2 or MueLu.
I'm running drekar on a single KNL of mutrino, with 1 MPI process, and I increase the OMP threads from 1 to 64 (1 OMP thread per core):
setup smoother (ifpack2 "MT Gauss-Seidel")
| t | solve time(s) |GS setup time(s)|
| -:| -:|-------:|
|1|33.27|493.10|
|2|24.67|286.50|
|4|12.26|157.80|
|8|6.97|79.82|
|16|3.97|36.61|
|32|3.50|24.06|
|64|3.16|16.01|
For reference, here are the times if I use the standard, non-threaded Gauss-Seidel
(but if it really is non-threaded, why is the setup time going down as the number of OMP threads is increased?)
setup smoother (ifpack2 "Gauss-Seidel")
|t|solve time(s)|GS setup time(s)|
| -:| -:|-------:|
|1|27.04|0.36|
|2|25.13|0.21|
|4|24.09|0.13|
|8|23.58|0.09|
|16|23.38|0.06|
|32|23.32|0.05|
|64|23.33|0.05|
drekar/Trilinos was built with intel 17.0.2 and gnu 6.1.0 (Trilinos repo as of August 16, 2017)
I ran vtune on ellis for the 1 OMP case (the 493.1s case above).
According to vtune, all the time is the two Kokkos::parallel_for calls in
KokkosKernels::Experimental::Util::symmetrize_graph_symbolic_hashmap (lines 1097 and 1139 of KokkosKernels_Utils.hpp)
the time is pretty much equally split between the two Kokkos::parallel_for calls
The following is the stack trace from Ifpack2:
```
Ifpack2::Relaxation::initialize()
KokkosKernels::Experimental::Graph::gauss_seidel_symbolic
KokkosKernels::Experimental::Graph::Impl::GaussSeidel
KokkosKernels::Experimental::Util::symmetrize_graph_symbolic_hashmap (Kokkos::parallel_for on line 1097 and 1139)
Kokkos::parallel_for
Kokkos::parallel_for
```
Edit (@aprokop): formattinghttps://gitlab.osti.gov/jmwille/Trilinos/-/issues/1401Experimental RBILUK 2017-09-21T16:05:25ZJames WillenbringExperimental RBILUK *Created by: mndevec*
This issue will track the development of new RBILUK.
Goals:
1- Clear separation of the symbolic, numeric and solve phases.
2- Replacing old blas calls such as GEMM, GETF2, with the register blocking little bl...*Created by: mndevec*
This issue will track the development of new RBILUK.
Goals:
1- Clear separation of the symbolic, numeric and solve phases.
2- Replacing old blas calls such as GEMM, GETF2, with the register blocking little block implementations in KokkosKernels.
The issue will be developed in the fork:
https://github.com/mndevec/Trilinos/tree/develop
@trilinos/ifpack2 https://gitlab.osti.gov/jmwille/Trilinos/-/issues/1423KokkosKernels: Problems with Kokkos_Sparse_CrsMatrix2017-07-03T19:41:58ZJames WillenbringKokkosKernels: Problems with Kokkos_Sparse_CrsMatrix*Created by: brian-kelley*
I'm trying to use the Kokkos::CrsMatrix to write a local threaded matrix addition. I've run into a couple of issues with it (one I've figured out, and one I haven't - maybe I should make 2 issues?):
-The do...*Created by: brian-kelley*
I'm trying to use the Kokkos::CrsMatrix to write a local threaded matrix addition. I've run into a couple of issues with it (one I've figured out, and one I haven't - maybe I should make 2 issues?):
-The documentation says that the [raw host array constructor](https://trilinos.org/docs/dev/packages/kokkos/doc/html/classKokkos_1_1CrsMatrix.html#a0c2a800a9ffc10f658038bb64ac47972) expects coordinates format (rows, cols, values) but it really expects CRS (rowptrs, colinds, values). I've changed my usage code to give CRS and it now works.
-I should be able to get a [Kokkos::SparseRowView](https://trilinos.org/docs/dev/packages/kokkos/doc/html/structKokkos_1_1SparseRowView.html) of row i by calling mat.row(i). This should let me directly get at the colinds and values in row i. But, every view I get back is empty (rowView.length == 0). I tried calling .row(i) for every row:
> C has 380473 nonzeros, 10000 rows, 10000 cols.
> Number of nz in each row: 0 0 0 0 0 0 0 0 0 0 (...and so on for 10,000 zeros)
C->nnz(), C->numRows() and C->numCols() all print what I expect (the first line of output), so there are entries in there. This looks like a simple bug, but is there anything special I need to do to get these row views?https://gitlab.osti.gov/jmwille/Trilinos/-/issues/1237KokkosKernels CI build failure on 4/12/20172017-04-12T16:26:22ZJames WillenbringKokkosKernels CI build failure on 4/12/2017*Created by: bartlettroscoe*
A KokkosKernels build failure was caught by the standard CI build:
* http://testing.sandia.gov/cdash/index.php?project=Trilinos&filtercount=3&showfilters=1&filtercombine=and&field1=buildname&compare1=61&v...*Created by: bartlettroscoe*
A KokkosKernels build failure was caught by the standard CI build:
* http://testing.sandia.gov/cdash/index.php?project=Trilinos&filtercount=3&showfilters=1&filtercombine=and&field1=buildname&compare1=61&value1=Linux-GCC-4.7.2-MPI_RELEASE_DEBUG_SHARED_PT_CI&field2=groupname&compare2=61&value2=Continuous&field3=buildstarttime&compare3=84&value3=now
in the first CI iteration this morning:
* http://testing.sandia.gov/cdash/index.php?project=Trilinos&parentid=2834087&filtercount=3&showfilters=1&field1=groupname&compare1=61&value1=Continuous&field2=buildstarttime&compare2=84&value2=now&filtercombine=and
(which suggests the checkin-test-sems.sh script should have caught this build failure before the push). But this was only a test/example failure which does not disable KokkosKernels in downstream packages. (Note that we will lose this behavior once we do the single configure, build, and tests https://github.com/TriBITSPub/TriBITS/issues/183 ).
But it looks like @crtrott reacted to this quickly fixed the KokkosKernels in the latest CI build iteration:
* http://testing.sandia.gov/cdash/index.php?project=Trilinos&parentid=2834513&filtercount=3&showfilters=1&field1=groupname&compare1=61&value1=Continuous&field2=buildstarttime&compare2=84&value2=now&filtercombine=and
Note that since this was only a test/example failure for KokkosKernels, this would not have stopped anyone else's pushes using the checkin-test-sems.sh script where were only changing a package downstream from KokkosKernels.
Just added this issue to document this failure since it was interesting for a few reasons:
* The build failure should have likely have been caused by the checkin-test-sems.sh script
* The failure was seen and fixed in the very next CI iteration
* This was only a build failure for examples/tests so this would not have blocked anyone else's push to changes in packages downstream from KokkosKernels
Since Trilinos CDash info is forgotten pretty quickly (just a month or so).
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/1180Consider taking advantage of new CUDA language support in CMake 3.8.02017-09-13T22:41:43ZJames WillenbringConsider taking advantage of new CUDA language support in CMake 3.8.0*Created by: bartlettroscoe*
**Description:**
I was looking over the release notes for CMake 3.8 at:
* https://cmake.org/cmake/help/v3.8/release/3.8.html
and it says that CUDA is now a first-class CMake language and it supports...*Created by: bartlettroscoe*
**Description:**
I was looking over the release notes for CMake 3.8 at:
* https://cmake.org/cmake/help/v3.8/release/3.8.html
and it says that CUDA is now a first-class CMake language and it supports the nvcc compiler.
Should this be something that we look into for Trilinos?
**CC:** @trilinos/framework, @trilinos/tpetra, @crtrott, @nmhamster
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/1176Analysis of configure and build failures due to KokkosKernels pushes on March...2017-03-23T17:54:16ZJames WillenbringAnalysis of configure and build failures due to KokkosKernels pushes on March 1-2 reported in #1099*Created by: bartlettroscoe*
**Description:**
This story is to analyze the KokkosKernels commits pushed on March 1-2, 2017 that broke the configure and build of Trilinos that was reported in #1099 and see if usage of the [checkin-tes...*Created by: bartlettroscoe*
**Description:**
This story is to analyze the KokkosKernels commits pushed on March 1-2, 2017 that broke the configure and build of Trilinos that was reported in #1099 and see if usage of the [checkin-test-sems.sh](https://github.com/trilinos/Trilinos/wiki/Policies-%7C-Safe-Checkin-Testing) script could have avoided the failures (and resulting consequences) if it had been used for the test and push (which would have stopped the push).
This started when a set of commits were pushed to the Trilinos 'develop' branch on March 1 with the top commit 97ed757 being:
```
97ed757 [Wed Mar 1 08:24:01 2017 -0700] <crtrott@sandia.gov>
Kokkos-Kernels: Adding Kokkos-Kernels as a stand-alone package
```
as shown by the CI build:
* http://testing.sandia.gov/cdash/viewConfigure.php?buildid=2766164
That version of Trilinos failed to configure as shown on that CI build iteration.
Later that day, issue #1099 was created by an important Trilinos customer and it resulted in 35 comments that involved 9 people in that issue before it was resolved.
An attempt to fix this problem was pushed later that day with the top commit de7ac5a being:
```
de7ac5a [Wed Mar 1 13:17:16 2017 -0700] <mhoemme@sandia.gov>
KokkosKernels: Fix #1099
```
as shown by the CI build:
* http://testing.sandia.gov/cdash/viewConfigure.php?buildid=2766462
That version passed the configure but resulted in many build failures.
The build was not finally fixed until March 2 as shown at:
* http://testing.sandia.gov/cdash/viewConfigure.php?buildid=2767860
Could the usage of the checkin-test-sems.sh script have caught these problems and stop the pushes that broke the configure and build of Trilinos over these two days?
**CC:** @trilinos/framework, @bathmatt, @crtrott, @mhoemmen
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/1170KokkosKernels: warnings on CUDA2017-10-27T04:19:03ZJames WillenbringKokkosKernels: warnings on CUDA*Created by: rppawlo*
When compiling with Tpetra enabled for cuda, we get a ton of warnings from kokkos kernels. It seems to stem from the three lines below. Can someone take a look at this?
```
[ 69%] Built target locathyra
/ascldap...*Created by: rppawlo*
When compiling with Tpetra enabled for cuda, we get a ton of warnings from kokkos kernels. It seems to stem from the three lines below. Can someone take a look at this?
```
[ 69%] Built target locathyra
/ascldap/users/rppawlo/Trilinos/packages/kokkos-kernels/src/Kokkos_ArithTraits.hpp(1480): warning: pointless comparison of unsigned integer with zero
/ascldap/users/rppawlo/Trilinos/packages/kokkos-kernels/src/Kokkos_ArithTraits.hpp(157): warning: pointless comparison of unsigned integer with zero
detected during instantiation of "IntType <unnamed>::intPowSigned(IntType, IntType) [with IntType=char]"
(1506): here
/ascldap/users/rppawlo/Trilinos/packages/kokkos-kernels/src/Kokkos_ArithTraits.hpp(161): warning: pointless comparison of unsigned integer with a negative constant
detected during instantiation of "IntType <unnamed>::intPowSigned(IntType, IntType) [with IntType=char]"
(1506): here
```https://gitlab.osti.gov/jmwille/Trilinos/-/issues/1099KokkosKernels pullout broke my configures2017-03-05T22:12:00ZJames WillenbringKokkosKernels pullout broke my configures*Created by: bathmatt*
Any idea on what I need to add to my scripts to get configure to work. It is a silent failure, by that I mean it isn't obvious what the problem is, and my CMakeErrors.log looks pretty much the same before and aft...*Created by: bathmatt*
Any idea on what I need to add to my scripts to get configure to work. It is a silent failure, by that I mean it isn't obvious what the problem is, and my CMakeErrors.log looks pretty much the same before and after the commit.
I added Trilinos_ENABLE_KokkosKernels=ON
Set up for creating a distribution ...
tail of the configure.
`
Finished configuring Trilinos!
-- Configuring incomplete, errors occurred!
See also "/jenkins/slave/workspace/EMPIRE-debug/build_drekar/CMakeFiles/CMakeOutput.log".
See also "/jenkins/slave/workspace/EMPIRE-debug/build_drekar/CMakeFiles/CMakeError.log".
Error in configuration of drekar, trying to continue`
Here is my configure script
[config.txt](https://github.com/trilinos/Trilinos/files/811733/config.txt)
and all the log files. log has been renamed to txt
[configure.txt](https://github.com/trilinos/Trilinos/files/811735/configure.txt)
[CMakeError.txt](https://github.com/trilinos/Trilinos/files/811737/CMakeError.txt)
[CMakeOutput.txt](https://github.com/trilinos/Trilinos/files/811738/CMakeOutput.txt)
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/1084Graph coloring build warnings2017-02-22T17:26:25ZJames WillenbringGraph coloring build warnings*Created by: mhoemmen*
This is a duplicate of https://github.com/kokkos/kokkos-kernels/issues/3 (not yet externally visible, but will be soon) for the sake of external tracking.*Created by: mhoemmen*
This is a duplicate of https://github.com/kokkos/kokkos-kernels/issues/3 (not yet externally visible, but will be soon) for the sake of external tracking.https://gitlab.osti.gov/jmwille/Trilinos/-/issues/988KokkosKernels: Gauss-Seidel test fails with CUDA2017-04-13T18:44:14ZJames WillenbringKokkosKernels: Gauss-Seidel test fails with CUDA*Created by: mhoemmen*
@trilinos/tpetra @trilinos/ifpack2
The Gauss-Seidel test that I reenabled as part of #977 fails in a CUDA debug build for me, with CUDA 7.5 and GCC 4.8.4. It passes in a non-CUDA Clang 3.9 build with Kokkos::...*Created by: mhoemmen*
@trilinos/tpetra @trilinos/ifpack2
The Gauss-Seidel test that I reenabled as part of #977 fails in a CUDA debug build for me, with CUDA 7.5 and GCC 4.8.4. It passes in a non-CUDA Clang 3.9 build with Kokkos::Serial. The error messages look like this:
```
result_norm_res:3.05254e+36 initial_norm_res:160.873
3: .../Trilinos/packages/tpetra/kernels/unit_test/graph/KokkosKernels_Test_GaussSeidel.cpp:219: Failure
3: Value of: (result_norm_res < initial_norm_res)
3: Actual: false
3: Expected: true
```
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/981Remove Tribits dependencies in KokkosKernels2017-01-06T03:27:05ZJames WillenbringRemove Tribits dependencies in KokkosKernels*Created by: srajama1*
There are bunch of tribits related dependencies that has come into KokkosKernels. Eventually we would like a Kokkos model where we could build without Tribits and test with gtest. This needs a cleanup.
@mndevec...*Created by: srajama1*
There are bunch of tribits related dependencies that has come into KokkosKernels. Eventually we would like a Kokkos model where we could build without Tribits and test with gtest. This needs a cleanup.
@mndevec @crtrott @mhoemmen
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/980KokkosKernels: Sparse matrix-matrix multiply performance test build error2017-04-13T18:43:39ZJames WillenbringKokkosKernels: Sparse matrix-matrix multiply performance test build error*Created by: mhoemmen*
@trilinos/ifpack2 @trilinos/tpetra
Lots of errors like this:
```
.../Trilinos/packages/tpetra/kernels/perf_test/graph/KokkosKernels_SPMM_KK.cpp: In function ‘int main(int, char**)’:
.../Trilinos/packages/tpet...*Created by: mhoemmen*
@trilinos/ifpack2 @trilinos/tpetra
Lots of errors like this:
```
.../Trilinos/packages/tpetra/kernels/perf_test/graph/KokkosKernels_SPMM_KK.cpp: In function ‘int main(int, char**)’:
.../Trilinos/packages/tpetra/kernels/perf_test/graph/KokkosKernels_SPMM_KK.cpp:688:13: error: ‘Kokkos::OpenMP’ has not been declared
```https://gitlab.osti.gov/jmwille/Trilinos/-/issues/979KokkosKernels: Graph coloring test fails2017-04-13T18:43:15ZJames WillenbringKokkosKernels: Graph coloring test fails*Created by: mhoemmen*
@trilinos/tpetra @trilinos/ifpack2
The KokkosKernels graph coloring test fails with the following run-time error:
```
terminate called after throwing an instance of 'std::runtime_error'
11: what(): Kokkos:...*Created by: mhoemmen*
@trilinos/tpetra @trilinos/ifpack2
The KokkosKernels graph coloring test fails with the following run-time error:
```
terminate called after throwing an instance of 'std::runtime_error'
11: what(): Kokkos::OpenMP parallel_for ERROR: not initialized
11: Traceback functionality not available
```
This looks pretty easy to fix. Somewhere, something is not initializing Kokkos::OpenMP.
I am disabling the test for now to ensure that I can proceed with #288.
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/978KokkosKernels: Sparse matrix-matrix multiply test build error2017-04-13T18:42:58ZJames WillenbringKokkosKernels: Sparse matrix-matrix multiply test build error*Created by: mhoemmen*
@trilinos/tpetra @trilinos/ifpack2
KokkosKernels' sparse matrix-matrix multiply test has a build error:
```
.../Trilinos/packages/tpetra/kernels/src/stage/graph/utils/KokkosKernels_SimpleUtils.hpp:50:41: error:...*Created by: mhoemmen*
@trilinos/tpetra @trilinos/ifpack2
KokkosKernels' sparse matrix-matrix multiply test has a build error:
```
.../Trilinos/packages/tpetra/kernels/src/stage/graph/utils/KokkosKernels_SimpleUtils.hpp:50:41: error: no match for ‘operator>’ (operand types are ‘Kokkos::ViewTraits<Kokkos::complex<double>*, Kokkos::LayoutRight, Kokkos::Device<Kokkos::OpenMP, Kokkos::HostSpace>, Kokkos::MemoryTraits<0u> >::value_type {aka Kokkos::complex<double>}’ and ‘int’)
#define KOKKOSKERNELS_MACRO_ABS(x) ((x) > (0) ? (x): (-x))
^
.../Trilinos/packages/tpetra/kernels/src/stage/graph/utils/KokkosKernels_SimpleUtils.hpp:133:9: note: in expansion of macro ‘KOKKOSKERNELS_MACRO_ABS’
if (KOKKOSKERNELS_MACRO_ABS (val_diff) > eps) {
...
```
This may just be an issue with the test not accounting for complex numbers.https://gitlab.osti.gov/jmwille/Trilinos/-/issues/977KokkosKernels: Gauss-Seidel test build error2017-01-06T21:13:24ZJames WillenbringKokkosKernels: Gauss-Seidel test build error*Created by: mhoemmen*
@trilinos/tpetra @trilinos/ifpack2
KokkosKernels' Gauss-Seidel test has a build error:
```
.../Trilinos/packages/tpetra/kernels/unit_test/graph/KokkosKernels_Test_GaussSeidel.cpp: In instantiation of ‘scalar_v...*Created by: mhoemmen*
@trilinos/tpetra @trilinos/ifpack2
KokkosKernels' Gauss-Seidel test has a build error:
```
.../Trilinos/packages/tpetra/kernels/unit_test/graph/KokkosKernels_Test_GaussSeidel.cpp: In instantiation of ‘scalar_view_t create_x_vector(size_t, typename scalar_view_t::value_type) [with scalar_view_t = Kokkos::View<Kokkos::complex<double>*, Kokkos::LayoutRight, Kokkos::Device<Kokkos::OpenMP, Kokkos::HostSpace>, Kokkos::MemoryTraits<0u> >; size_t = long unsigned int; typename scalar_view_t::value_type = Kokkos::complex<double>]’:
.../Trilinos/packages/tpetra/kernels/unit_test/graph/KokkosKernels_Test_GaussSeidel.cpp:178:66: required from ‘void test_gauss_seidel(KokkosKernels::Experimental::Graph::GSAlgorithm) [with scalar_t = Kokkos::complex<double>; lno_t = int; device = Kokkos::Device<Kokkos::OpenMP, Kokkos::HostSpace>]’
.../Trilinos/packages/tpetra/kernels/unit_test/graph/KokkosKernels_Test_GaussSeidel.cpp:232:3: required from here
.../Trilinos/packages/tpetra/kernels/unit_test/graph/KokkosKernels_Test_GaussSeidel.cpp:140:68: error: no match for ‘operator/’ (operand types are ‘int’ and ‘Kokkos::ViewTraits<Kokkos::complex<double>*, Kokkos::LayoutRight, Kokkos::Device<Kokkos::OpenMP, Kokkos::HostSpace>, Kokkos::MemoryTraits<0u> >::value_type {aka Kokkos::complex<double>}’)
static_cast <typename scalar_view_t::value_type> (RAND_MAX / max_value);
^
```
I think the code itself is fine; at least it builds and runs correctly in Ifpack2. This looks like an issue with the test not accounting for complex numbers.https://gitlab.osti.gov/jmwille/Trilinos/-/issues/948KokkosKernels: SPMV performance comparison test2017-01-06T00:12:10ZJames WillenbringKokkosKernels: SPMV performance comparison test*Created by: crtrott*
This adds a performance comparison test without MPI for just SPMV kernel. Can compare the KokkosKernel function, a custom Kokkos implementation, CuSparse and MKL.
This can read MatrixMarket format, and also gener...*Created by: crtrott*
This adds a performance comparison test without MPI for just SPMV kernel. Can compare the KokkosKernel function, a custom Kokkos implementation, CuSparse and MKL.
This can read MatrixMarket format, and also generate a binary matrix storage file, which it can read again. Depending on your system reading the binary format back in can be 10x faster than the text file.
I had that written quite a while ago I just cleaned it up now.