Trilinos issueshttps://gitlab.osti.gov/jmwille/Trilinos/-/issues2017-03-08T17:07:48Zhttps://gitlab.osti.gov/jmwille/Trilinos/-/issues/1110Teko with MueLu error2017-03-08T17:07:48ZJames WillenbringTeko with MueLu error*Created by: bathmatt*
I'm trying to get a handle and get started with this error. when I try to enable Teko using MueLu as the preconditioner for one of the block solves I get an error. This all works with ML.
Asking @egphill @eri...*Created by: bathmatt*
I'm trying to get a handle and get started with this error. when I try to enable Teko using MueLu as the preconditioner for one of the block solves I get an error. This all works with ML.
Asking @egphill @eric-c-cyr @jhux2 if they've seen this before? I'm thinking I need special options inmy configure?
Here is the cryptic error message. I'm thinking it may be a configure issue?
```
Clearing old data (if any)
Teko: "buildInverse" could not construct the inverse operator using "Thyra::MueLuPreconditionerFactory"
*** THROWN EXCEPTION ***
std::bad_cast
************************
Teko: "buildInverse" could not construct the inverse operator using "Teko::AutoClone<drekar::blkprec::PlasmaPreconditionerFactory, drekar::blkprec::PlasmaPreconditionerFactory>"
*** THROWN EXCEPTION ***
std::exception
************************
*********** Caught Exception std::exception: Begin Error Report ***********
std::exception
************ Caught Exception std::exception: End Error Report ************
=========================================================================================================================================================================
```https://gitlab.osti.gov/jmwille/Trilinos/-/issues/1109checkin-test-sems.sh: how to control -j for make and ctest ?2017-03-06T16:18:22ZJames Willenbringcheckin-test-sems.sh: how to control -j for make and ctest ?*Created by: ibaned*
I'm trying to give separate `-j` arguments to `make` and `ctest` through `checkin-test-sems.sh`. This is the way I'm trying to do it:
```bash
../cmake/std/sems/checkin-test-sems.sh \
--do-all --make-options="-j16...*Created by: ibaned*
I'm trying to give separate `-j` arguments to `make` and `ctest` through `checkin-test-sems.sh`. This is the way I'm trying to do it:
```bash
../cmake/std/sems/checkin-test-sems.sh \
--do-all --make-options="-j16" --ctest-options="-j1"
```
But that runs the following make command:
```bash
make -j4 -j16
```
How do I make the first `-j4` disappear ?https://gitlab.osti.gov/jmwille/Trilinos/-/issues/1108Amesos Umfpack Parallel Hang2017-06-20T18:20:08ZJames WillenbringAmesos Umfpack Parallel Hang*Created by: tgvoskuilen*
I have been experiencing intermittent parallel hangs on multi-processor jobs using umfpack (solver block below). I've run it in the debugger and it's just sitting at two different MPI calls within the umfpack s...*Created by: tgvoskuilen*
I have been experiencing intermittent parallel hangs on multi-processor jobs using umfpack (solver block below). I've run it in the debugger and it's just sitting at two different MPI calls within the umfpack solver code. I captured matrix files for the case where it hangs and can provide them too.
begin trilinos equation solver directVerbose
solution method = amesos-umfpack
fei output level = matrix_files
endhttps://gitlab.osti.gov/jmwille/Trilinos/-/issues/1107panzer on cuda 7.5.182017-03-07T14:33:35ZJames Willenbringpanzer on cuda 7.5.18*Created by: bathmatt*
On shiller I cannot compile panzer with cuda, but can with my desktop. The major difference is cuda 8 vs 7.5.18.
@rppawlo @mhoemmen @crtrott do you have any ideas why these would be different?
Are we using Ko...*Created by: bathmatt*
On shiller I cannot compile panzer with cuda, but can with my desktop. The major difference is cuda 8 vs 7.5.18.
@rppawlo @mhoemmen @crtrott do you have any ideas why these would be different?
Are we using Kokkos::Compat::KokkosDeviceWrapperNode anymore?
There are two different errors
Which looks like panzer::TpetraNodeType != Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Cuda, Kokkos::Cuda::memory_space>>
```
[100%] Building CXX object packages/panzer/dof-mgr/src/CMakeFiles/panzer-dof-mgr.dir/Panzer_Filtered_UniqueGlobalIndexer.cpp.o
/home/mbetten/workspace/Trilinos/packages/teuchos/core/src/Teuchos_RCP.hpp(288): error: a value of type "const Tpetra::Map<int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Cuda, Kokkos::Cuda::memory_space>> *" cannot be used to initialize an entity of type "const Tpetra::Map<int, int, panzer::TpetraNodeType> *"
detected during:
instantiation of "Teuchos::RCP<T>::RCP(const Teuchos::RCP<T2> &) [with T=const Tpetra::Map<int, int, panzer::TpetraNodeType>, T2=const Tpetra::Map<int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Cuda, Kokkos::Cuda::memory_space>>]"
/home/mbetten/workspace/Trilinos/packages/panzer/dof-mgr/src/Panzer_Filtered_UniqueGlobalIndexer_impl.hpp(116): here
instantiation of "void panzer::Filtered_UniqueGlobalIndexer<LocalOrdinalT, GlobalOrdinalT>::getOwnedAndGhostedNotFilteredIndicator(std::vector<int, std::allocator<int>> &) const [with LocalOrdinalT=int, GlobalOrdinalT=int]"
/home/mbetten/workspace/Trilinos/packages/panzer/dof-mgr/src/Panzer_Filtered_UniqueGlobalIndexer.cpp(48): here
1 error detected in the compilation of "/tmp/tmpxft_0000854e_00000000-7_Panzer_Filtered_UniqueGlobalIndexer.cpp1.ii".
make[2]: *** [packages/panzer/dof-mgr/src/CMakeFiles/panzer-dof-mgr.dir/Panzer_Filtered_UniqueGlobalIndexer.cpp.o] Error 2
```
The second error looks like view incompatibility, It could be the same thing.
```
[100%] Building CXX object packages/panzer/dof-mgr/src/CMakeFiles/panzer-dof-mgr.dir/Panzer_DOFManager.cpp.o
/home/mbetten/workspace/Trilinos/packages/teuchos/core/src/Teuchos_RCP.hpp(288): error: a value of type "const Tpetra::Map<int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Cuda, Kokkos::Cuda::memory_space>> *" cannot be used to initialize an entity of type "const Tpetra::Map<int, int, panzer::TpetraNodeType> *"
detected during:
instantiation of "Teuchos::RCP<T>::RCP(const Teuchos::RCP<T2> &) [with T=const Tpetra::Map<int, int, panzer::TpetraNodeType>, T2=const Tpetra::Map<int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Cuda, Kokkos::Cuda::memory_space>>]"
/home/mbetten/workspace/Trilinos/packages/panzer/dof-mgr/src/Panzer_DOFManager_impl.hpp(1164): here
instantiation of "Teuchos::RCP<const Tpetra::Map<LocalOrdinalT, GlobalOrdinalT, panzer::TpetraNodeType>> panzer::DOFManager<LocalOrdinalT, GlobalOrdinalT>::buildOverlapMapFromElements(const panzer::DOFManager<LocalOrdinalT, GlobalOrdinalT>::ElementBlockAccess &) const [with LocalOrdinalT=int, GlobalOrdinalT=int]"
/home/mbetten/workspace/Trilinos/packages/panzer/dof-mgr/src/Panzer_DOFManager_impl.hpp(396): here
/home/mbetten/workspace/Trilinos/packages/kokkos/core/src/Kokkos_View.hpp(1221): error: static assertion failed with "Incompatible View copy construction"
detected during:
instantiation of "Kokkos::View<DataType, Properties...>::View(const Kokkos::View<RT, RP...> &) [with DataType=int **, Properties=<Kokkos::LayoutLeft, Kokkos::Cuda::execution_space, void>, RT=int **, RP=<Kokkos::LayoutLeft, PHX::Device>]"
/home/mbetten/workspace/Trilinos/packages/panzer/dof-mgr/src/Panzer_DOFManager_impl.hpp(662): here
instantiation of "std::pair<Teuchos::RCP<Tpetra::MultiVector<GlobalOrdinalT, LocalOrdinalT, GlobalOrdinalT, panzer::TpetraNodeType, false>>, Teuchos::RCP<Tpetra::MultiVector<GlobalOrdinalT, LocalOrdinalT, GlobalOrdinalT, panzer::TpetraNodeType, false>>> panzer::DOFManager<LocalOrdinalT, GlobalOrdinalT>::buildGlobalUnknowns_GUN(const Tpetra::MultiVector<GlobalOrdinalT, LocalOrdinalT, GlobalOrdinalT, panzer::TpetraNodeType, false> &, Tpetra::MultiVector<GlobalOrdinalT, LocalOrdinalT, GlobalOrdinalT, panzer::TpetraNodeType, false> &) const [with LocalOrdinalT=int, GlobalOrdinalT=int]"
/home/mbetten/workspace/Trilinos/packages/panzer/dof-mgr/src/Panzer_DOFManager_impl.hpp(377): here
/home/mbetten/workspace/Trilinos/packages/kokkos/core/src/impl/Kokkos_ViewMapping.hpp(2884): error: static assertion failed with "View assignment must have compatible spaces"
detected during:
instantiation of "void Kokkos::Experimental::Impl::ViewMapping<DstTraits, SrcTraits, std::enable_if<<expression>, void>::type>::assign(Kokkos::Experimental::Impl::ViewMapping<DstTraits, SrcTraits, std::enable_if<<expression>, void>::type>::DstType &, const Kokkos::Experimental::Impl::ViewMapping<DstTraits, SrcTraits, std::enable_if<<expression>, void>::type>::SrcType &, const Kokkos::Experimental::Impl::ViewMapping<DstTraits, SrcTraits, std::enable_if<<expression>, void>::type>::TrackType &) [with DstTraits=Kokkos::ViewTraits<int **, Kokkos::LayoutLeft, Kokkos::Cuda::execution_space, void>, SrcTraits=Kokkos::ViewTraits<int **, Kokkos::LayoutLeft, PHX::Device>]"
/home/mbetten/workspace/Trilinos/packages/kokkos/core/src/Kokkos_View.hpp(1222): here
instantiation of "Kokkos::View<DataType, Properties...>::View(const Kokkos::View<RT, RP...> &) [with DataType=int **, Properties=<Kokkos::LayoutLeft, Kokkos::Cuda::execution_space, void>, RT=int **, RP=<Kokkos::LayoutLeft, PHX::Device>]"
/home/mbetten/workspace/Trilinos/packages/panzer/dof-mgr/src/Panzer_DOFManager_impl.hpp(662): here
instantiation of "std::pair<Teuchos::RCP<Tpetra::MultiVector<GlobalOrdinalT, LocalOrdinalT, GlobalOrdinalT, panzer::TpetraNodeType, false>>, Teuchos::RCP<Tpetra::MultiVector<GlobalOrdinalT, LocalOrdinalT, GlobalOrdinalT, panzer::TpetraNodeType, false>>> panzer::DOFManager<LocalOrdinalT, GlobalOrdinalT>::buildGlobalUnknowns_GUN(const Tpetra::MultiVector<GlobalOrdinalT, LocalOrdinalT, GlobalOrdinalT, panzer::TpetraNodeType, false> &, Tpetra::MultiVector<GlobalOrdinalT, LocalOrdinalT, GlobalOrdinalT, panzer::TpetraNodeType, false> &) const [with LocalOrdinalT=int, GlobalOrdinalT=int]"
/home/mbetten/workspace/Trilinos/packages/panzer/dof-mgr/src/Panzer_DOFManager_impl.hpp(377): here
```https://gitlab.osti.gov/jmwille/Trilinos/-/issues/1106EpetraExt examples not integrated2017-03-03T19:03:31ZJames WillenbringEpetraExt examples not integrated*Created by: MicheldeMessieres*
EpetraExt has examples which are not integrated into the normal Trilinos examples structure. PR #1105 adds one of them (HDF5_IO.cpp). It was decided not to add all of them at this time. This issue is to t...*Created by: MicheldeMessieres*
EpetraExt has examples which are not integrated into the normal Trilinos examples structure. PR #1105 adds one of them (HDF5_IO.cpp). It was decided not to add all of them at this time. This issue is to track the need to incorporate other examples for testing purposes.https://gitlab.osti.gov/jmwille/Trilinos/-/issues/1104ctest -j and kokkos and hwloc2017-03-21T01:55:23ZJames Willenbringctest -j and kokkos and hwloc*Created by: bathmatt*
@nmhamster @rppawlo
Is there any procedure on how to test in parallel on the various systems
https://github.com/kokkos/kokkos/issues/630
points out an issue with openmp and thread binding.
I'm looking for...*Created by: bathmatt*
@nmhamster @rppawlo
Is there any procedure on how to test in parallel on the various systems
https://github.com/kokkos/kokkos/issues/630
points out an issue with openmp and thread binding.
I'm looking for a recipe on what do I configure with, and how do I test for the different platforms. Particularly for openmpi/RHEL, ellis, ride, shiller (cpu and gpu).
What are people using? Are you just reverting to -j1?
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/1103MueLu builds zero column prolongator for diagonal matrix2017-05-22T17:41:47ZJames WillenbringMueLu builds zero column prolongator for diagonal matrix*Created by: tawiesn*
By default, MueLu drops "Dirichlet" nodes from aggregation. In the worst case it builds a prolongation operator with zero columns.
There is a "aggregation: preserve Dirichlet points" option (default=off) to keep...*Created by: tawiesn*
By default, MueLu drops "Dirichlet" nodes from aggregation. In the worst case it builds a prolongation operator with zero columns.
There is a "aggregation: preserve Dirichlet points" option (default=off) to keep all Dirichlet nodes unaggregated. The problem is, that there would be no progress in coarsening if we activate that.
What are the options for a smarter handling of this special case?
- We could detect the problem during the aggregation (all nodes "aggregated" with 0 aggregates) and then build a dummy one node aggregate (the coarsening would terminate on the next level)
- We check the size of two consecutive levels and if the matrix size is not shrinking (by a factor X) we stop coarsening...https://gitlab.osti.gov/jmwille/Trilinos/-/issues/1102Zoltan multilevel hypergraph partitioning doesn't generate enough levels on a...2017-03-31T21:51:49ZJames WillenbringZoltan multilevel hypergraph partitioning doesn't generate enough levels on a network with communities.*Created by: rsln-s*
I'm running zoltan phg on a matrix generated from [youtube dataset with ground-truth communities](http://socialnetworks.mpi-sws.org/data-imc2007.html) in a following way: communities are hyperedges (rows) and users ...*Created by: rsln-s*
I'm running zoltan phg on a matrix generated from [youtube dataset with ground-truth communities](http://socialnetworks.mpi-sws.org/data-imc2007.html) in a following way: communities are hyperedges (rows) and users are vertices (i.e. a vertex is in a hyperedge if as user is a member of the corresponding community)
However, on that matrix with ~660k vertices zoltan only calls coarsening twice and therefore only generates 3 levels with ~590k vertices on the lowest one, despite the fact that I've specifically set `PHG_COARSENING_LIMIT = 1000`
Matrix and zdrive.inp files are attached. `.txt` had to be appended to the filenames so that github would let me upload them.
[zdrive.inp.youtube.agg.2.txt](https://github.com/trilinos/Trilinos/files/814424/zdrive.inp.youtube.agg.2.txt)
[youtube.mtx.txt](https://github.com/trilinos/Trilinos/files/814427/youtube.mtx.txt)
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/1101Kokkos_Sparse_impl_MKL build error2017-03-03T15:27:03ZJames WillenbringKokkos_Sparse_impl_MKL build error*Created by: kongm*
Hi,
I'm trying to build the latest Trilinos git version. I'm using these options:
-D TSQR_ENABLE_Fortran:BOOL=OFF \
-D Trilinos_ENABLE_Belos:BOOL=ON \
-D Trilinos_ENABLE_Ifpack2:BOOL=ON \
-D Trilinos...*Created by: kongm*
Hi,
I'm trying to build the latest Trilinos git version. I'm using these options:
-D TSQR_ENABLE_Fortran:BOOL=OFF \
-D Trilinos_ENABLE_Belos:BOOL=ON \
-D Trilinos_ENABLE_Ifpack2:BOOL=ON \
-D Trilinos_ENABLE_Teuchos:BOOL=ON \
-D Trilinos_ENABLE_CXX11:BOOL=ON \
-D Trilinos_ENABLE_Tpetra:BOOL=ON \
-D Trilinos_ENABLE_Kokkos:BOOL=ON \
-D Trilinos_ENABLE_KokkosKernels:BOOL=ON \
-D Trilinos_ENABLE_OpenMP:BOOL=ON \
-D Trilinos_ENABLE_KokkosKernels:BOOL=ON \
-D Trilinos_ENABLE_Epetra:BOOL=OFF \
-D Trilinos_ENABLE_TpetraKernels:BOOL=ON \
-D Trilinos_ENABLE_DEVELOPMENT_MODE:BOOL=OFF \
-D Trilinos_ENABLE_Panzer:BOOL=OFF \
-D Trilinos_ENABLE_DEBUG:BOOL=OFF \
-D Trilinos_ENABLE_Thyra:BOOL=OFF \
-D Trilinos_ENABLE_CHECKED_STL:BOOL=OFF \
-D Trilinos_ENABLE_ML:BOOL=OFF \
-D Tpetra_INST_OPENMP:BOOL=ON \
-D Tpetra_PRINT_Efficiency_Warnings:BOOL=ON \
-D Tpetra_ENABLE_EXAMPLES:BOOL=OFF \
-D Tpetra_ENABLE_TESTS:BOOL=OFF \
-D Tpetra_ENABLE_Kernels:BOOL=OFF \
-D Teuchos_ENABLE_COMPLEX:BOOL=OFF \
-D Teuchos_ENABLE_EXAMPLES:BOOL=OFF \
-D Teuchos_ENABLE_DEBUG_RCP_NODE_TRACING:BOOL=OFF \
-D Kokkos_ENABLE_OpenMP:BOOL=ON \
-D Kokkos_ENABLE_EXAMPLES:BOOL=OFF \
-D Kokkos_ENABLE_TESTS:BOOL=OFF \
-D Kokkos_ENABLE_TSQR:BOOL=OFF \
-D Kokkos_ENABLE_BOUNDS_CHECK:BOOL=ON \
-D Kokkos_ENABLE_Serial:BOOL=ON \
-D Belos_ENABLE_TEUCHOS_TIME_MONITOR:BOOL=ON \
The configuration process completes successfully, but when building I get errors such as:
In function ‘std::string KokkosSparse::Impl::Mkl::matrixDescriptorToString(const int&, std::string, bool, bool)’:
/glb/home/usmkpb/mkongdir/trilinos-src-git/packages/tpetra/kernels/src/impl/Kokkos_Sparse_impl_MKL.cpp:200:64: error: request for member ‘type’ in ‘descr’, which is of non-class type ‘const int’
out << indent << "Type: " << sparseMatrixTypeToString (descr.type);
or
In function ‘std::string KokkosSparse::Impl::Mkl::sparseFillModeToString(int)’:
/glb/home/usmkpb/mkongdir/trilinos-src-git/packages/tpetra/kernels/src/impl/Kokkos_Sparse_impl_MKL.cpp:127:15: error: ‘SPARSE_FILL_MODE_LOWER’ was not declared in this scope
if (mode == SPARSE_FILL_MODE_LOWER) {
as well as other error about not finding sparse_matrix_t, status_type, sparse_status_t, etc.
Am I missing some flags?
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/1100Panzer compile tutorials under cuda2017-03-24T15:31:24ZJames WillenbringPanzer compile tutorials under cuda*Created by: bathmatt*
@rppawlo
I assume the mySourceTerm is your stuff? It doesn't compile under cuda due to the functor vs lambda stuff. *Created by: bathmatt*
@rppawlo
I assume the mySourceTerm is your stuff? It doesn't compile under cuda due to the functor vs lambda stuff. https://gitlab.osti.gov/jmwille/Trilinos/-/issues/1099KokkosKernels pullout broke my configures2017-03-05T22:12:00ZJames WillenbringKokkosKernels pullout broke my configures*Created by: bathmatt*
Any idea on what I need to add to my scripts to get configure to work. It is a silent failure, by that I mean it isn't obvious what the problem is, and my CMakeErrors.log looks pretty much the same before and aft...*Created by: bathmatt*
Any idea on what I need to add to my scripts to get configure to work. It is a silent failure, by that I mean it isn't obvious what the problem is, and my CMakeErrors.log looks pretty much the same before and after the commit.
I added Trilinos_ENABLE_KokkosKernels=ON
Set up for creating a distribution ...
tail of the configure.
`
Finished configuring Trilinos!
-- Configuring incomplete, errors occurred!
See also "/jenkins/slave/workspace/EMPIRE-debug/build_drekar/CMakeFiles/CMakeOutput.log".
See also "/jenkins/slave/workspace/EMPIRE-debug/build_drekar/CMakeFiles/CMakeError.log".
Error in configuration of drekar, trying to continue`
Here is my configure script
[config.txt](https://github.com/trilinos/Trilinos/files/811733/config.txt)
and all the log files. log has been renamed to txt
[configure.txt](https://github.com/trilinos/Trilinos/files/811735/configure.txt)
[CMakeError.txt](https://github.com/trilinos/Trilinos/files/811737/CMakeError.txt)
[CMakeOutput.txt](https://github.com/trilinos/Trilinos/files/811738/CMakeOutput.txt)
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/1098MueLu fails to compile on cuda2017-03-07T22:02:01ZJames WillenbringMueLu fails to compile on cuda*Created by: bathmatt*
I'm seeing these errors when I try to compile MueLu
`
[ 87%] Building CXX object packages/muelu/src/CMakeFiles/muelu.dir/Utils/ExplicitInstantiation/MueLu_CoalesceDropFactory.cpp.o
/home/mbetten/Trilinos/Tril...*Created by: bathmatt*
I'm seeing these errors when I try to compile MueLu
`
[ 87%] Building CXX object packages/muelu/src/CMakeFiles/muelu.dir/Utils/ExplicitInstantiation/MueLu_CoalesceDropFactory.cpp.o
/home/mbetten/Trilinos/Trilinos/packages/xpetra/src/Map/Xpetra_EpetraMap.hpp(655): warning: statement is unreachable
/home/mbetten/Trilinos/Trilinos/packages/xpetra/src/CrsGraph/Xpetra_CrsGraphFactory.hpp(114): error: object of abstract class type "Xpetra::TpetraCrsGraph<int, int, Xpetra::EpetraNode>" is not allowed:
pure virtual function "Xpetra::CrsGraph<LocalOrdinal, GlobalOrdinal, Node>::computeGlobalConstants [with LocalOrdinal=int, GlobalOrdinal=int, Node=Xpetra::EpetraNode]" has no overrider
/home/mbetten/Trilinos/Trilinos/packages/xpetra/src/CrsGraph/Xpetra_CrsGraphFactory.hpp(114): error: no instance of overloaded function "Xpetra::rcp" matches the argument list
argument types are: (<error-type>)
`
Here is a compile line, long as it is
`cd /home/mbetten/builds/drekar/cuda/debug/packages/muelu/src && /projects/sems/install/rhel6-x86_64/sems/compiler/gcc/5.3.0/openmpi/1.10.1/bin/mpicxx -I/home/mbetten/builds/drekar/cuda/debug -I/home/mbetten/Trilinos/Trilinos/packages/muelu/src -I/home/mbetten/Trilinos/Trilinos/packages/muelu/src/Graph -I/home/mbetten/Trilinos/Trilinos/packages/muelu/src/Graph/BrickAggregation -I/home/mbetten/Trilinos/Trilinos/packages/muelu/src/Graph/UncoupledAggregation -I/home/mbetten/Trilinos/Trilinos/packages/muelu/src/Graph/UserAggregation -I/home/mbetten/Trilinos/Trilinos/packages/muelu/src/Headers -I/home/mbetten/Trilinos/Trilinos/packages/muelu/src/Interface -I/home/mbetten/Trilinos/Trilinos/packages/muelu/src/Interface/FacadeClasses -I/home/mbetten/Trilinos/Trilinos/packages/muelu/src/Misc -I/home/mbetten/Trilinos/Trilinos/packages/muelu/src/MueCentral -I/home/mbetten/Trilinos/Trilinos/packages/muelu/src/Rebalancing -I/home/mbetten/Trilinos/Trilinos/packages/muelu/src/Smoothers -I/home/mbetten/Trilinos/Trilinos/packages/muelu/src/Transfers/BaseClass -I/home/mbetten/Trilinos/Trilinos/packages/muelu/src/Smoothers/BlockedSmoothers -I/home/mbetten/Trilinos/Trilinos/packages/muelu/src/Transfers/BlockedTransfers -I/home/mbetten/Trilinos/Trilinos/packages/muelu/src/Transfers/Generic -I/home/mbetten/Trilinos/Trilinos/packages/muelu/src/Transfers/Energy-Minimization -I/home/mbetten/Trilinos/Trilinos/packages/muelu/src/Transfers/Energy-Minimization/Solvers -I/home/mbetten/Trilinos/Trilinos/packages/muelu/src/Transfers/Geometric-Interpolation -I/home/mbetten/Trilinos/Trilinos/packages/muelu/src/Transfers/Smoothed-Aggregation -I/home/mbetten/Trilinos/Trilinos/packages/muelu/src/Transfers/Petrov-Galerkin-SA -I/home/mbetten/Trilinos/Trilinos/packages/muelu/src/Transfers/User -I/home/mbetten/Trilinos/Trilinos/packages/muelu/src/Transfers/SemiCoarsen -I/home/mbetten/Trilinos/Trilinos/packages/muelu/src/Utils -I/home/mbetten/Trilinos/Trilinos/packages/muelu/src/Utils/ForwardDeclaration -I/home/mbetten/Trilinos/Trilinos/packages/muelu/src/../adapters/epetra -I/home/mbetten/Trilinos/Trilinos/packages/muelu/src/../adapters/tpetra -I/home/mbetten/Trilinos/Trilinos/packages/muelu/src/Transfers/PCoarsen -I/home/mbetten/builds/drekar/cuda/debug/packages/muelu/src -I/home/mbetten/builds/drekar/cuda/debug/packages/intrepid2/refactor/src -I/home/mbetten/Trilinos/Trilinos/packages/intrepid2/refactor/src/Cell -I/home/mbetten/Trilinos/Trilinos/packages/intrepid2/refactor/src/Discretization/Basis -I/home/mbetten/Trilinos/Trilinos/packages/intrepid2/refactor/src/Discretization/FunctionSpaceTools -I/home/mbetten/Trilinos/Trilinos/packages/intrepid2/refactor/src/Discretization/Integration -I/home/mbetten/Trilinos/Trilinos/packages/intrepid2/refactor/src/Orientation -I/home/mbetten/Trilinos/Trilinos/packages/intrepid2/refactor/src/Shared -I/home/mbetten/Trilinos/Trilinos/packages/intrepid2/refactor/src -I/home/mbetten/builds/drekar/cuda/debug/packages/shards/src -I/home/mbetten/Trilinos/Trilinos/packages/shards/src -I/home/mbetten/builds/drekar/cuda/debug/packages/sacado/src -I/home/mbetten/Trilinos/Trilinos/packages/sacado/src -I/home/mbetten/Trilinos/Trilinos/packages/sacado/src/template -I/home/mbetten/Trilinos/Trilinos/packages/sacado/src/parameter -I/home/mbetten/Trilinos/Trilinos/packages/sacado/src/mpl -I/home/mbetten/builds/drekar/cuda/debug/packages/teuchos/kokkoscomm/src -I/home/mbetten/Trilinos/Trilinos/packages/teuchos/kokkoscomm/src -I/home/mbetten/builds/drekar/cuda/debug/packages/teuchos/kokkoscompat/src -I/home/mbetten/Trilinos/Trilinos/packages/teuchos/kokkoscompat/src -I/home/mbetten/Trilinos/Trilinos/packages/teuchos/parameterlist/src -I/home/mbetten/builds/drekar/cuda/debug/packages/teuchos/core/src -I/home/mbetten/Trilinos/Trilinos/packages/teuchos/core/src -I/home/mbetten/builds/drekar/cuda/debug/packages/kokkos/core/src -I/home/mbetten/Trilinos/Trilinos/packages/kokkos/core/src -I/usr/local/cuda/include -I/projects/sems/install/rhel6-x86_64/sems/tpl/boost/1.59.0/gcc/5.3.0/base/include -I/projects/sems/install/rhel6-x86_64/sems/tpl/yaml_cpp/0.5.3/gcc/5.3.0/base/include -I/home/mbetten/Trilinos/Trilinos/packages/teuchos/comm/src -I/home/mbetten/Trilinos/Trilinos/packages/teuchos/remainder/src -I/home/mbetten/builds/drekar/cuda/debug/packages/teuchos/remainder/src -I/home/mbetten/Trilinos/Trilinos/packages/teuchos/numerics/src -I/home/mbetten/builds/drekar/cuda/debug/packages/kokkos/algorithms/src -I/home/mbetten/Trilinos/Trilinos/packages/kokkos/algorithms/src -I/home/mbetten/builds/drekar/cuda/debug/packages/kokkos/containers/src -I/home/mbetten/Trilinos/Trilinos/packages/kokkos/containers/src -I/home/mbetten/builds/drekar/cuda/debug/packages/teko/src -I/home/mbetten/Trilinos/Trilinos/packages/teko/src -I/home/mbetten/Trilinos/Trilinos/packages/teko/src/NS -I/home/mbetten/Trilinos/Trilinos/packages/teko/src/Epetra -I/home/mbetten/Trilinos/Trilinos/packages/teko/src/Tpetra -I/home/mbetten/builds/drekar/cuda/debug/packages/stratimikos/src -I/home/mbetten/Trilinos/Trilinos/packages/stratimikos/src -I/home/mbetten/Trilinos/Trilinos/packages/stratimikos/adapters/belos/src -I/home/mbetten/builds/drekar/cuda/debug/packages/stratimikos/adapters/belos/src -I/home/mbetten/Trilinos/Trilinos/packages/stratimikos/adapters/aztecoo/src -I/home/mbetten/Trilinos/Trilinos/packages/stratimikos/adapters/amesos/src -I/home/mbetten/Trilinos/Trilinos/packages/stratimikos/adapters/ml/src -I/home/mbetten/Trilinos/Trilinos/packages/stratimikos/adapters/ifpack/src -I/home/mbetten/builds/drekar/cuda/debug/packages/belos/tpetra/src -I/home/mbetten/Trilinos/Trilinos/packages/belos/tpetra/src -I/home/mbetten/builds/drekar/cuda/debug/packages/belos/epetra/src -I/home/mbetten/Trilinos/Trilinos/packages/belos/epetra/src -I/home/mbetten/builds/drekar/cuda/debug/packages/belos/src -I/home/mbetten/Trilinos/Trilinos/packages/belos/src -I/home/mbetten/builds/drekar/cuda/debug/packages/aztecoo/src -I/home/mbetten/Trilinos/Trilinos/packages/aztecoo/src -I/home/mbetten/builds/drekar/cuda/debug/packages/triutils/src -I/home/mbetten/Trilinos/Trilinos/packages/triutils/src -I/home/mbetten/builds/drekar/cuda/debug/packages/epetra/src -I/home/mbetten/Trilinos/Trilinos/packages/epetra/src -I/home/mbetten/Trilinos/Trilinos/packages/thyra/adapters/tpetra/src -I/home/mbetten/Trilinos/Trilinos/packages/thyra/adapters/epetra/src -I/home/mbetten/Trilinos/Trilinos/packages/thyra/core/src -I/home/mbetten/Trilinos/Trilinos/packages/thyra/core/src/interfaces/operator_vector/fundamental -I/home/mbetten/Trilinos/Trilinos/packages/thyra/core/src/interfaces/operator_vector/extended -I/home/mbetten/Trilinos/Trilinos/packages/thyra/core/src/support/operator_vector/client_support -I/home/mbetten/Trilinos/Trilinos/packages/thyra/core/src/support/operator_vector/adapter_support -I/home/mbetten/Trilinos/Trilinos/packages/thyra/core/src/interfaces/operator_solve/fundamental -I/home/mbetten/Trilinos/Trilinos/packages/thyra/core/src/interfaces/operator_solve/extended -I/home/mbetten/Trilinos/Trilinos/packages/thyra/core/src/support/operator_solve/client_support -I/home/mbetten/Trilinos/Trilinos/packages/thyra/core/src/interfaces/nonlinear/model_evaluator/fundamental -I/home/mbetten/Trilinos/Trilinos/packages/thyra/core/src/support/nonlinear/model_evaluator/client_support -I/home/mbetten/Trilinos/Trilinos/packages/thyra/core/src/interfaces/nonlinear/solvers/fundamental -I/home/mbetten/Trilinos/Trilinos/packages/thyra/core/src/support/nonlinear/solvers/client_support -I/home/mbetten/builds/drekar/cuda/debug/packages/thyra/core/src -I/home/mbetten/Trilinos/Trilinos/packages/thyra/core/example/operator_vector -I/home/mbetten/Trilinos/Trilinos/packages/rtop/src -I/home/mbetten/Trilinos/Trilinos/packages/rtop/src/interfaces -I/home/mbetten/Trilinos/Trilinos/packages/rtop/src/support -I/home/mbetten/Trilinos/Trilinos/packages/rtop/src/ops_lib -I/home/mbetten/Trilinos/Trilinos/packages/rtop/src/lapack -I/home/mbetten/builds/drekar/cuda/debug/packages/rtop/src -I/home/mbetten/Trilinos/Trilinos/packages/tpetra/core/ext -I/home/mbetten/builds/drekar/cuda/debug/packages/tpetra/core/ext -I/home/mbetten/Trilinos/Trilinos/packages/tpetra/core/inout -I/home/mbetten/builds/drekar/cuda/debug/packages/tpetra/core/inout -I/home/mbetten/Trilinos/Trilinos/packages/tpetra/core/src -I/home/mbetten/Trilinos/Trilinos/packages/tpetra/core/src/kokkos_refactor -I/home/mbetten/builds/drekar/cuda/debug/packages/tpetra/core/src -I/home/mbetten/builds/drekar/cuda/debug/packages/tpetra/kernels/src -I/home/mbetten/Trilinos/Trilinos/packages/tpetra/kernels/src -I/home/mbetten/Trilinos/Trilinos/packages/tpetra/kernels/src/impl -I/home/mbetten/Trilinos/Trilinos/packages/tpetra/kernels/src/stage/graph -I/home/mbetten/Trilinos/Trilinos/packages/tpetra/kernels/src/stage/graph/impl -I/home/mbetten/Trilinos/Trilinos/packages/tpetra/kernels/src/stage/graph/utils -I/home/mbetten/Trilinos/Trilinos/packages/tpetra/classic/LinAlg -I/home/mbetten/Trilinos/Trilinos/packages/tpetra/classic/NodeAPI -I/home/mbetten/builds/drekar/cuda/debug/packages/tpetra/classic/NodeAPI -I/home/mbetten/builds/drekar/cuda/debug/packages/tpetra/classic/src -I/home/mbetten/Trilinos/Trilinos/packages/tpetra/classic/src -I/home/mbetten/Trilinos/Trilinos/packages/thyra/adapters/epetraext/src/model_evaluator -I/home/mbetten/Trilinos/Trilinos/packages/thyra/adapters/epetraext/src/transformer -I/home/mbetten/builds/drekar/cuda/debug/packages/epetraext/src -I/home/mbetten/Trilinos/Trilinos/packages/epetraext/src -I/home/mbetten/Trilinos/Trilinos/packages/epetraext/src/transform -I/home/mbetten/Trilinos/Trilinos/packages/epetraext/src/inout -I/home/mbetten/Trilinos/Trilinos/packages/epetraext/src/coloring -I/home/mbetten/Trilinos/Trilinos/packages/epetraext/src/model_evaluator -I/home/mbetten/Trilinos/Trilinos/packages/epetraext/src/block -I/home/mbetten/Trilinos/Trilinos/packages/epetraext/src/restrict -I/home/mbetten/builds/drekar/cuda/debug/packages/ml/src -I/home/mbetten/Trilinos/Trilinos/packages/ml/src/Include -I/home/mbetten/Trilinos/Trilinos/packages/ml/src/Comm -I/home/mbetten/Trilinos/Trilinos/packages/ml/src/Coarsen -I/home/mbetten/Trilinos/Trilinos/packages/ml/src/FEGrid -I/home/mbetten/Trilinos/Trilinos/packages/ml/src/Main -I/home/mbetten/Trilinos/Trilinos/packages/ml/src/Smoother -I/home/mbetten/Trilinos/Trilinos/packages/ml/src/Krylov -I/home/mbetten/Trilinos/Trilinos/packages/ml/src/Operator -I/home/mbetten/Trilinos/Trilinos/packages/ml/src/Utils -I/home/mbetten/Trilinos/Trilinos/packages/ml/src/RefMaxwell -I/home/mbetten/Trilinos/Trilinos/packages/ml/src/MatrixFree -I/home/mbetten/Trilinos/Trilinos/packages/ml/src/MLAPI -I/home/mbetten/Trilinos/Trilinos/packages/ml/src/LevelWrap -I/home/mbetten/builds/drekar/cuda/debug/packages/ifpack/src -I/home/mbetten/Trilinos/Trilinos/packages/ifpack/src -I/home/mbetten/Trilinos/Trilinos/common/auxiliarySoftware/SuiteSparse/src/AMD/Include -I/home/mbetten/Trilinos/Trilinos/common/auxiliarySoftware/SuiteSparse/src/UFconfig -I/home/mbetten/builds/drekar/cuda/debug/packages/amesos/src -I/home/mbetten/Trilinos/Trilinos/packages/amesos/src -I/home/mbetten/Trilinos/Trilinos/packages/amesos/src/SuiteSparse/AMD/Include -I/home/mbetten/Trilinos/Trilinos/packages/amesos/src/SuiteSparse/COLAMD/Include -I/home/mbetten/Trilinos/Trilinos/packages/amesos/src/SuiteSparse/UFconfig -I/home/mbetten/Trilinos/Trilinos/packages/amesos/src/SuiteSparse/KLU/Include -I/home/mbetten/Trilinos/Trilinos/packages/amesos/src/SuiteSparse/BTF/Include -I/home/mbetten/Trilinos/Trilinos/packages/amesos/src/SuiteSparse/CAMD/Include -I/home/mbetten/Trilinos/Trilinos/packages/amesos/src/SuiteSparse/CCOLAMD/Include -I/home/mbetten/Trilinos/Trilinos/packages/common/auxiliarySoftware/SuiteSparse/src/AMD/Include -I/home/mbetten/Trilinos/Trilinos/packages/common/auxiliarySoftware/SuiteSparse/src/BTF/Include -I/home/mbetten/Trilinos/Trilinos/packages/common/auxiliarySoftware/SuiteSparse/src/CAMD/Include -I/home/mbetten/Trilinos/Trilinos/packages/common/auxiliarySoftware/SuiteSparse/src/CCOLAMD/Include -I/home/mbetten/Trilinos/Trilinos/packages/common/auxiliarySoftware/SuiteSparse/src/COLAMD/Include -I/home/mbetten/Trilinos/Trilinos/packages/common/auxiliarySoftware/SuiteSparse/src/KLU/Include -I/home/mbetten/Trilinos/Trilinos/packages/common/auxiliarySoftware/SuiteSparse/src/UFconfig -I/home/mbetten/builds/drekar/cuda/debug/packages/zoltan/src -I/home/mbetten/Trilinos/Trilinos/packages/zoltan/src/include -I/home/mbetten/Trilinos/Trilinos/packages/zoltan/src/all -I/home/mbetten/Trilinos/Trilinos/packages/zoltan/src/coloring -I/home/mbetten/Trilinos/Trilinos/packages/zoltan/src/graph -I/home/mbetten/Trilinos/Trilinos/packages/zoltan/src/ha -I/home/mbetten/Trilinos/Trilinos/packages/zoltan/src/hier -I/home/mbetten/Trilinos/Trilinos/packages/zoltan/src/hsfc -I/home/mbetten/Trilinos/Trilinos/packages/zoltan/src/lb -I/home/mbetten/Trilinos/Trilinos/packages/zoltan/src/matrix -I/home/mbetten/Trilinos/Trilinos/packages/zoltan/src/order -I/home/mbetten/Trilinos/Trilinos/packages/zoltan/src/par -I/home/mbetten/Trilinos/Trilinos/packages/zoltan/src/params -I/home/mbetten/Trilinos/Trilinos/packages/zoltan/src/tpls -I/home/mbetten/Trilinos/Trilinos/packages/zoltan/src/phg -I/home/mbetten/Trilinos/Trilinos/packages/zoltan/src/rcb -I/home/mbetten/Trilinos/Trilinos/packages/zoltan/src/reftree -I/home/mbetten/Trilinos/Trilinos/packages/zoltan/src/simple -I/home/mbetten/Trilinos/Trilinos/packages/zoltan/src/timer -I/home/mbetten/Trilinos/Trilinos/packages/zoltan/src/Utilities/Communication -I/home/mbetten/Trilinos/Trilinos/packages/zoltan/src/Utilities/DDirectory -I/home/mbetten/Trilinos/Trilinos/packages/zoltan/src/Utilities/Timer -I/home/mbetten/Trilinos/Trilinos/packages/zoltan/src/Utilities/shared -I/home/mbetten/Trilinos/Trilinos/packages/zoltan/src/zz -I/home/mbetten/Trilinos/Trilinos/packages/ifpack2/adapters/thyra -I/home/mbetten/builds/drekar/cuda/debug/packages/ifpack2/adapters -I/home/mbetten/Trilinos/Trilinos/packages/ifpack2/src -I/home/mbetten/builds/drekar/cuda/debug/packages/ifpack2/src -I/home/mbetten/Trilinos/Trilinos/packages/zoltan2/src -I/home/mbetten/builds/drekar/cuda/debug/packages/zoltan2/src -I/home/mbetten/Trilinos/Trilinos/packages/zoltan2/src/algorithms -I/home/mbetten/Trilinos/Trilinos/packages/zoltan2/src/algorithms/partition -I/home/mbetten/Trilinos/Trilinos/packages/zoltan2/src/algorithms/order -I/home/mbetten/Trilinos/Trilinos/packages/zoltan2/src/algorithms/color -I/home/mbetten/Trilinos/Trilinos/packages/zoltan2/src/algorithms/match -I/home/mbetten/Trilinos/Trilinos/packages/zoltan2/src/algorithms/map -I/home/mbetten/Trilinos/Trilinos/packages/zoltan2/src/algorithms/zoltan -I/home/mbetten/Trilinos/Trilinos/packages/zoltan2/src/environment -I/home/mbetten/Trilinos/Trilinos/packages/zoltan2/src/include -I/home/mbetten/Trilinos/Trilinos/packages/zoltan2/src/input -I/home/mbetten/Trilinos/Trilinos/packages/zoltan2/src/models -I/home/mbetten/Trilinos/Trilinos/packages/zoltan2/src/problems -I/home/mbetten/Trilinos/Trilinos/packages/zoltan2/src/util -I/home/mbetten/Trilinos/Trilinos/packages/xpetra/sup/Matrix -I/home/mbetten/Trilinos/Trilinos/packages/xpetra/sup/StridedMap -I/home/mbetten/Trilinos/Trilinos/packages/xpetra/sup/Cloner -I/home/mbetten/Trilinos/Trilinos/packages/xpetra/sup/Utils -I/home/mbetten/builds/drekar/cuda/debug/packages/xpetra/sup -I/home/mbetten/Trilinos/Trilinos/packages/xpetra/src/CrsGraph -I/home/mbetten/Trilinos/Trilinos/packages/xpetra/src/CrsMatrix -I/home/mbetten/Trilinos/Trilinos/packages/xpetra/src/BlockedCrsMatrix -I/home/mbetten/Trilinos/Trilinos/packages/xpetra/src/DistObject -I/home/mbetten/Trilinos/Trilinos/packages/xpetra/src/Export -I/home/mbetten/Trilinos/Trilinos/packages/xpetra/src/Headers -I/home/mbetten/Trilinos/Trilinos/packages/xpetra/src/Import -I/home/mbetten/Trilinos/Trilinos/packages/xpetra/src/Map -I/home/mbetten/Trilinos/Trilinos/packages/xpetra/src/BlockedMap -I/home/mbetten/Trilinos/Trilinos/packages/xpetra/src/MultiVector -I/home/mbetten/Trilinos/Trilinos/packages/xpetra/src/BlockedMultiVector -I/home/mbetten/Trilinos/Trilinos/packages/xpetra/src/BlockedVector -I/home/mbetten/Trilinos/Trilinos/packages/xpetra/src/Operator -I/home/mbetten/Trilinos/Trilinos/packages/xpetra/src/Platform -I/home/mbetten/Trilinos/Trilinos/packages/xpetra/src/RowGraph -I/home/mbetten/Trilinos/Trilinos/packages/xpetra/src/RowMatrix -I/home/mbetten/Trilinos/Trilinos/packages/xpetra/src/Utils -I/home/mbetten/Trilinos/Trilinos/packages/xpetra/src/Utils/ForwardDeclaration -I/home/mbetten/Trilinos/Trilinos/packages/xpetra/src/Vector -I/home/mbetten/builds/drekar/cuda/debug/packages/xpetra/src -I/home/mbetten/builds/drekar/cuda/debug/packages/anasazi/tpetra/src -I/home/mbetten/Trilinos/Trilinos/packages/anasazi/tpetra/src -I/home/mbetten/builds/drekar/cuda/debug/packages/anasazi/epetra/util/ModeLaplace -I/home/mbetten/Trilinos/Trilinos/packages/anasazi/epetra/util/ModeLaplace -I/home/mbetten/builds/drekar/cuda/debug/packages/anasazi/epetra/src -I/home/mbetten/Trilinos/Trilinos/packages/anasazi/epetra/src -I/home/mbetten/builds/drekar/cuda/debug/packages/anasazi/src -I/home/mbetten/Trilinos/Trilinos/packages/anasazi/src -I/home/mbetten/Trilinos/Trilinos/packages/anasazi/src/../thyra/src -I/home/mbetten/Trilinos/Trilinos/packages/anasazi/src/../tpetra/src -std=c++11 -Wno-unused-local-typedefs -g -O0 -o CMakeFiles/muelu.dir/Utils/ExplicitInstantiation/MueLu_CoalesceDropFactory.cpp.o -c /home/mbetten/builds/drekar/cuda/debug/packages/muelu/src/Utils/ExplicitInstantiation/MueLu_CoalesceDropFactory.cpp
`https://gitlab.osti.gov/jmwille/Trilinos/-/issues/1097TeuchosComm Requires MPI v2 but this is not tested for at compile time.2017-03-22T04:30:52ZJames WillenbringTeuchosComm Requires MPI v2 but this is not tested for at compile time.*Created by: jjellio*
Reported by David Hysom,
`MPI_Comm_create_keyval` is an MPI2 function, if a user attempts to build with MPI-1, they will get atleast the error below (potentially others?). Is there a compile time check for the M...*Created by: jjellio*
Reported by David Hysom,
`MPI_Comm_create_keyval` is an MPI2 function, if a user attempts to build with MPI-1, they will get atleast the error below (potentially others?). Is there a compile time check for the MPI spec the compiler supports? If not, attempting to compile a program with `MPI_Comm_create_keyval` could be an option.
```
packages/teuchos/comm/src/Teuchos_MpiReductionOpSetter.cpp:171:43: error: 'MPI_Comm_create_keyval' was not declared in this scope
&key, NULL);
```
@trilinos/teuchos @trilinos/framework https://gitlab.osti.gov/jmwille/Trilinos/-/issues/1095Zoltan2 Complains about ParMETIS Version (4.0.3) but this is Installed?2017-04-04T17:08:38ZJames WillenbringZoltan2 Complains about ParMETIS Version (4.0.3) but this is Installed?*Created by: nmhamster*
@kddevin @vjleung - seem to be having some trouble with Zoltan2 on the Bowman KNL test bed on Trilinos VOTD for 2017-FEB-26.
I am getting an error during CMake configure:
```
Processing enabled package: Zo...*Created by: nmhamster*
@kddevin @vjleung - seem to be having some trouble with Zoltan2 on the Bowman KNL test bed on Trilinos VOTD for 2017-FEB-26.
I am getting an error during CMake configure:
```
Processing enabled package: Zoltan2 (Libs, Tests)
CMake Error at packages/zoltan2/CMakeLists.txt:8 (MESSAGE):
Zoltan2 requires ParMETIS_version 4.0.3 or later. Upgrade or set
TPL_ENABLE_ParMETIS OFF or set Zoltan2_ENABLE_ParMETIS OFF.
```
But I have the ParMETIS 4.0.3 module loaded and it is pointing to a 4.0.3 install.
Here is my configure script (see below) which I am running with the `devpack-sparc/openmpi/1.10.4/intel/17.0.098` module loaded (this should load everything we need including METIS, ParMETIS, SuperLUDist etc). Right now, I need to have ParMETIS working because SuperLUDist is required for the upstream customers.
I am probably missing something in the configure but, if that is the case, I guess this is not a good error message? I am going to proceed by disable ParMETIS for Zoltan2 and see if the build will continue. I guess one possible issue is that the code must cross compile successfully.
Appreciate any help or advice you might have on the configure.
```
INSTALL_DIR=/home/sdhammo/trilinos-sparc/20170226/openmpi/1.10.4/intel/17.0.098
ZLIB_DIR=${ZLIB_ROOT}
BOOST_ROOT=${BOOST_ROOT}
TRILINOS_HOME=/home/sdhammo/git/trilinos-github-repo
MPI_HOME=${MPI_ROOT}
HDF5_DIR=${HDF5_ROOT}
NETCDF_DIR=${NETCDF_ROOT}
PNETCDF_DIR=${PNETCDF_ROOT}
C_COMPILE_FLAGS="-O3 -g -xMIC-AVX512 -mkl=parallel -qopt-report=5"
FORTRAN_COMPILE_FLAGS="-O3 -g -xMIC-AVX512 -mkl=parallel -qopt-report=5"
LINK_FLAGS="-O3 -g -xMIC-AVX512 -ldl -mkl=parallel"
BUILD_OPENMP=ON
BUILD_PTHREADS=OFF
rm CMakeCache.txt
rm -fr CMakeFiles
cmake \
-D Trilinos_USE_ATTB_DEV_ENV:BOOL=OFF \
-D Trilinos_ENABLE_OpenMP:BOOL=$BUILD_OPENMP \
-D MPI_EXEC_POST_NUMPROCS_FLAGS:STRING="--map-by;numa:PE=4" \
-D Trilinos_ENABLE_EXPLICIT_INSTANTIATION:BOOL=ON \
-D Trilinos_ENABLE_INSTALL_CMAKE_CONFIG_FILES:BOOL=ON \
-D Trilinos_ENABLE_EXAMPLES:BOOL=OFF \
-D Trilinos_ENABLE_TESTS:BOOL=ON \
-D Trilinos_ENABLE_ALL_OPTIONAL_PACKAGES=OFF \
-D Trilinos_ASSERT_MISSING_PACKAGES=OFF \
-D Trilinos_ALLOW_NO_PACKAGES:BOOL=OFF \
-D Trilinos_ENABLE_ALL_PACKAGES:BOOL=OFF \
-D Trilinos_ENABLE_Shards:BOOL=ON \
-D Trilinos_ENABLE_Pamgen:BOOL=ON \
-D Pamgen_ENABLE_TESTS:BOOL=ON \
-D Pamgen_ENABLE_EXAMPLES:BOOL=ON \
-D Trilinos_ENABLE_Intrepid:BOOL=ON \
-D Intrepid_ENABLE_TESTS:BOOL=OFF \
-D Intrepid_ENABLE_EXAMPLES:BOOL=ON \
-D Trilinos_ENABLE_Tpetra:BOOL=ON \
-D Trilinos_ENABLE_Epetra:BOOL=ON \
-D Trilinos_ENABLE_ML:BOOL=ON \
-D Trilinos_ENABLE_MueLu:BOOL=ON \
-D Trilinos_ENABLE_EpetraExt:BOOL=ON \
-D Trilinos_ENABLE_Teuchos:BOOL=ON \
-D Trilinos_ENABLE_AztecOO:BOOL=ON \
-D Trilinos_ENABLE_Belos:BOOL=ON \
-D Trilinos_ENABLE_aristos:BOOL=OFF \
-D Trilinos_ENABLE_FEApp:BOOL=OFF \
-D Trilinos_ENABLE_Claps:BOOL=OFF \
-D Trilinos_ENABLE_Ifpack2:BOOL=ON \
-D Trilinos_ENABLE_Amesos2:BOOL=ON \
-D Trilinos_ENABLE_Zoltan2:BOOL=ON \
-D Trilinos_ENABLE_Ifpack:BOOL=ON \
-D Trilinos_ENABLE_Amesos:BOOL=ON \
-D Trilinos_ENABLE_Zoltan:BOOL=ON \
-D Trilinos_ENABLE_STKMesh:BOOL=ON \
-D Trilinos_ENABLE_STKIO:BOOL=ON \
-D Trilinos_ENABLE_STKTransfer:BOOL=ON \
-D Trilinos_ENABLE_STKSearch:BOOL=ON \
-D Trilinos_ENABLE_STKUtil:BOOL=ON \
-D Trilinos_ENABLE_STKTopology:BOOL=ON \
-D Trilinos_ENABLE_Gtest:BOOL=ON \
-D Trilinos_ENABLE_STKClassic:BOOL=OFF \
-D Trilinos_ENABLE_SEACASExodus:BOOL=ON \
-D Trilinos_ENABLE_SEACASEpu:BOOL=ON \
-D Trilinos_ENABLE_SEACASExodiff:BOOL=ON \
-D Trilinos_ENABLE_SEACASNemspread:BOOL=ON \
-D Trilinos_ENABLE_SEACASNemslice:BOOL=ON \
-D Trilinos_ENABLE_KokkosAlgorithms:BOOL=ON \
-D Trilinos_ENABLE_KokkosCore:BOOL=ON \
-D Trilinos_ENABLE_KokkosCompat:BOOL=ON \
-D Trilinos_ENABLE_KokkosLinAlg:BOOL=ON \
-D Trilinos_ENABLE_KokkosContainers:BOOL=ON \
-D Trilinos_ENABLE_KokkosAlgorithms:BOOL=ON \
-D Trilinos_ENABLE_KokkosMpiComm:BOOL=ON \
-D Trilinos_ENABLE_TriKota:BOOL=OFF \
-D Trilinos_ENABLE_PyTrilinos:BOOL=OFF \
-D Kokkos_ENABLE_Pthread:BOOL=$BUILD_PTHREADS \
-D Kokkos_ENABLE_OpenMP:BOOL=$BUILD_OPENMP \
-D Kokkos_ENABLE_HWLOC:BOOL=OFF \
-D Kokkos_ENABLE_Profiling:BOOL=ON \
-D Trilinos_ENABLE_Tpetra:BOOL=ON \
-D Tpetra_ENABLE_Kokkos_Refactor:BOOL=ON \
\
-D TPL_ENABLE_Netcdf:BOOL=ON \
-D Netcdf_INCLUDE_DIRS:FILEPATH="$NETCDF_DIR/include" \
-D Netcdf_LIBRARY_DIRS:FILEPATH="$NETCDF_DIR/lib" \
-D TPL_Netcdf_PARALLEL:BOOL=ON \
-D TPL_Netcdf_LIBRARIES:PATH="$NETCDF_DIR/lib/libnetcdf.a;$HDF5_DIR/lib/libhdf5_hl.a;$HDF5_DIR/lib/libhdf5.a;$ZLIB_DIR/lib/libz.a;$PNETCDF_DIR/lib/libpnetcdf.a" \
-D TPL_ENABLE_HDF5:BOOL=ON \
-D HDF5_INCLUDE_DIRS:FILEPATH="$HDF5_DIR/include" \
-D TPL_HDF5_LIBRARIES:PATH="$HDF5_DIR/lib/libhdf5_hl.a;$HDF5_DIR/lib/libhdf5.a;$ZLIB_DIR/lib/libz.a" \
-D TPL_ENABLE_Zlib:STRING=ON \
-D Zlib_INCLUDE_DIRS:PATH="$ZLIB_DIR/include" \
-D TPL_Zlib_LIBRARIES:PATH="$ZLIB_DIR/lib/libz.a" \
-D TPL_Netcdf_Enables_PNetcdf:BOOL=TRUE \
-D TPL_Netcdf_Enables_Netcdf4:BOOL=TRUE \
-D TPL_ENABLE_Pnetcdf:BOOL=ON \
-D Pnetcdf_LIBRARY_DIRS:STRING="$PNETCDF_DIR/lib" \
-D Pnetcdf_INCLUDE_DIRS:STRING="$PNETCDF_DIR/include" \
-D Tpetra_INST_COMPLEX_FLOAT:BOOL=OFF \
-D Tpetra_INST_COMPLEX_DOUBLE:BOOL=OFF \
-D Tpetra_INST_INT_LONG_LONG:BOOL=ON \
-D Tpetra_INST_INT_INT:BOOL=ON \
-D Tpetra_INST_INT_LONG:BOOL=ON \
\
-D TPL_ENABLE_HWLOC:BOOL=ON \
-D TPL_HWLOC_LIBRARIES:STRING="$HWLOC_ROOT/lib/libhwloc.a;$NUMA_ROOT/lib/libnuma.a" \
-D HWLOC_INCLUDE_DIRS:STRING="$HWLOC_ROOT/include" \
\
-D CMAKE_C_COMPILER="$MPI_HOME/bin/mpicc" \
-D CMAKE_CXX_COMPILER="$MPI_HOME/bin/mpicxx" \
-D CMAKE_Fortran_COMPILER="$MPI_HOME/bin/mpif90" \
-D CMAKE_CXX_FLAGS:STRING="$C_COMPILE_FLAGS" \
-D CMAKE_C_FLAGS:STRING="$C_COMPILE_FLAGS" \
-D CMAKE_Fortran_FLAGS:STRING="$FORTRAN_COMPILE_FLAGS" \
-D CMAKE_EXE_LINKER_FLAGS="$LINK_FLAGS" \
-D CMAKE_SKIP_RULE_DEPENDENCY=ON \
-D TPL_ENABLE_MPI:BOOL=ON \
-D MPI_BASE_DIR:PATH="$MPI_HOME" \
-D MPI_BIN_DIR:FILEPATH="$MPI_HOME/bin" \
-D MPI_EXEC_MAX_NUMPROCS:STRING=4 \
-D CMAKE_BUILD_TYPE=NONE \
-D CMAKE_INSTALL_PREFIX:PATH=${INSTALL_DIR} \
-D TPL_ENABLE_BLAS:BOOL=ON \
-D TPL_ENABLE_LAPACK:BOOL=ON \
-D Teuchos_ENABLE_COMPLEX:BOOL=OFF \
-D Teuchos_HAVE_TEUCHOS_LAPACKLARND:BOOL=OFF \
-D HAVE_TEUCHOS_LAPACKLARND:BOOL=OFF \
-D Teuchos_HAVE_CXX_PRAGMA_WEAK:BOOL=OFF \
-D Tpetra_INST_SERIAL:BOOL=ON \
-D TPL_BLAS_LIBRARIES:STRING="${MKLROOT}/lib/intel64/libmkl_intel_thread.a" \
-D TPL_LAPACK_LIBRARIES:STRING="${MKLROOT}/lib/intel64/libmkl_intel_thread.a" \
-D TPL_ENABLE_SuperLUDist:BOOL=ON \
-D SuperLUDist_INCLUDE_DIRS:PATH=${SUPERLUDIST_ROOT}/include \
-D TPL_SuperLUDist_LIBRARIES="${SUPERLUDIST_ROOT}/lib/libsuperlu_dist_4.3.a;${MKLROOT}/lib/intel64/libmkl_intel_thread.a" \
-D TPL_ENABLE_ParMETIS:BOOL=ON \
-D ParMETIS_INCLUDE_DIRS:PATH="${PARMETIS_ROOT}/include" \
-D ParMETIS_LIBRARY_DIRS:PATH="${PARMETIS_ROOT}/lib" \
-D TPL_ENABLE_METIS:BOOL=ON \
-D METIS_INCLUDE_DIRS:PATH="${METIS_ROOT}/include" \
-D METIS_LIBRARY_DIRS:PATH="${METIS_ROOT}/lib" \
-D TPL_ENABLE_BoostLib:BOOL=ON \
-D BoostLib_INCLUDE_DIRS:PATH="${BOOST_ROOT}/include" \
-D BoostLib_LIBRARY_DIRS:PATH="${BOOST_ROOT}/lib" \
-D TPL_ENABLE_Boost:BOOL=ON \
-D Boost_INCLUDE_DIRS:PATH="${BOOST_ROOT}/include" \
-D Boost_LIBRARY_DIRS:PATH="${BOOST_ROOT}/lib" \
-D HAVE_CXX_PRAGMA_WEAK:BOOL=OFF \
-D MueLu_ENABLE_EXAMPLES=ON \
${EXTRA-ARGS} \
${TRILINOS_HOME}
```https://gitlab.osti.gov/jmwille/Trilinos/-/issues/1094Muelu / IntrepidPCoarsenFactory tests depend on the serial execution space2017-02-25T22:30:30ZJames WillenbringMuelu / IntrepidPCoarsenFactory tests depend on the serial execution space*Created by: jjellio*
Hi!
I have a build that is OpenMP only, and the unit tests from `IntrepidPCoarsenFactory.cpp` want to explicitly use `Kokkos::Serial`.
This causes a compile time error, since the type is undefined. I've tempo...*Created by: jjellio*
Hi!
I have a build that is OpenMP only, and the unit tests from `IntrepidPCoarsenFactory.cpp` want to explicitly use `Kokkos::Serial`.
This causes a compile time error, since the type is undefined. I've temporarily disabled these.
It looks like the tests wants to ensure determinism:
` void testUniqueNumbering(Teuchos::FancyOStream &out, bool &success)`
This one looks to ensure things are numbered correctly... which should be good to test w/threads.
` void testBuildSampleElementToNodeMapThreeElementQuad(Teuchos::FancyOStream &out, bool &success)
`
I've temporarily changed them to use `Kokkos::DefaultExecutionSpace`, which should allow my build to finish.
@trilinos/intrepid2 @trilinos/muelu @kyungjoo-kim
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/1093MueLu, yaml-cpp, Boost, Unit test failures2017-02-25T08:11:12ZJames WillenbringMueLu, yaml-cpp, Boost, Unit test failures*Created by: jjellio*
When building with
TPLs yaml-cpp and boost, MueLu's unit tests will fail to build, because CMake will fail to find yaml-cpp files during configuration.
Should this tests be in MueLu still? Perhaps it belongs in...*Created by: jjellio*
When building with
TPLs yaml-cpp and boost, MueLu's unit tests will fail to build, because CMake will fail to find yaml-cpp files during configuration.
Should this tests be in MueLu still? Perhaps it belongs in Teuchos, or is it redundant?
packages/muelu/test/unit_tests/CMakeLists.txt has:
```
IF (TPL_ENABLE_yaml-cpp AND TPL_ENABLE_Boost)
INCLUDE_DIRECTORIES(${yaml-cpp_INCLUDE_DIRS})
INCLUDE_DIRECTORIES(${CMAKE_CURRENT_SOURCE_DIR}/../../src/Interface/YamlParameterList)
APPEND_SET(SOURCES yaml/YamlParameterList.cpp)
TRIBITS_COPY_FILES_TO_BINARY_DIR(MueLu_Yaml_cp
SOURCE_DIR ${MueLu_SOURCE_DIR}/test/unit_tests/yaml/
DEST_DIR ${CMAKE_CURRENT_BINARY_DIR}/yaml/
SOURCE_FILES Match1.yaml Match1.xml Match2.yaml Match2.xml Match3.yaml Match3.xml Match4.yaml Match4.xml Arrays.yaml IllegalKeyString.yaml IntVsDouble.xml IntVsDouble.yaml MueLuConfig.yaml MueLuConfig.xml InconsistentArray.yaml input_restingHydrostatic_RK4.xml plasma_oscillation_rtc.xml
)
ENDIF()
```
YamlParameterList lives in Teuchos now:
```
[jjellio@mutrino:~/src/Tril...nos-jjellio/packages] find -name "*YamlParameterList*"
./teuchos/comm/src/Teuchos_YamlParameterListHelpers.cpp
./teuchos/comm/src/Teuchos_YamlParameterListHelpers.hpp
./teuchos/parameterlist/src/Teuchos_YamlParameterListCoreHelpers.hpp
./teuchos/parameterlist/test/yaml/YamlParameterList.cpp
```
@trilinos/muelu @trilinos/teuchos https://gitlab.osti.gov/jmwille/Trilinos/-/issues/1092Stratimikos: Amesos2 SuperLU_DIST test failing2017-09-13T21:50:27ZJames WillenbringStratimikos: Amesos2 SuperLU_DIST test failing*Created by: ibaned*
This is an unusual issue in that the relevant code is not yet in Trilinos at the time of posting, but it will help us to track problems with PR #1090 .
If one checks out that code and compiles with Thyra, Tpetra, A...*Created by: ibaned*
This is an unusual issue in that the relevant code is not yet in Trilinos at the time of posting, but it will help us to track problems with PR #1090 .
If one checks out that code and compiles with Thyra, Tpetra, Amesos2, Stratimikos, KLU2, and SuperLUDist enabled, the following command:
```
ctest -VV -R Stratimikos_test_single_amesos2_tpetra_solver_driver_SuperLU_DIST_MPI_1
```
Produces the following output:
```
UpdateCTestConfiguration from :/home/daibane/build/host/Trilinos/DartConfiguration.tcl
Parse Config file:/home/daibane/build/host/Trilinos/DartConfiguration.tcl
Add coverage exclude regular expressions.
SetCTestConfiguration:CMakeCommand:/usr/local/bin/cmake
UpdateCTestConfiguration from :/home/daibane/build/host/Trilinos/DartConfiguration.tcl
Parse Config file:/home/daibane/build/host/Trilinos/DartConfiguration.tcl
Test project /home/daibane/build/host/Trilinos
Constructing a list of tests
Done constructing a list of tests
Checking test dependency graph...
Checking test dependency graph end
test 13
Start 13: Stratimikos_test_single_amesos2_tpetra_solver_driver_SuperLU_DIST_MPI_1
13: Test command: /home/daibane/install/host/mpich/bin/mpiexec "-np" "1" "/home/daibane/build/host/Trilinos/packages/stratimikos/adapters/amesos2/test/Stratimikos_test_single_amesos2_tpetra_solver_driver.exe" "--show-all-tests" "--solver-type=SuperLU_DIST" "--verbose" "--matrix-file=A.mm"
13: Test timeout computed to be: 1500
13: Teuchos::GlobalMPISession::GlobalMPISession(): started processor with name westley.srn.sandia.gov and rank 0!
13:
13: ***
13: *** Testing Thyra::BelosLinearOpWithSolveFactory (and Thyra::BelosLinearOpWithSolve)
13: ***
13:
13: Echoing input options:
13: matrixFile = A.mm
13: numRhs = 1
13: numRandomVectors = 1
13: maxFwdError = 1e-14
13: maxResid = 1e-06
13: showAllTests = 1
13: dumpAll = 0
13:
13: A) Reading in a tpetra matrix A from the file 'A.mm' ...
13:
13: B) Creating an Amesos2LinearOpWithSolveFactory object opFactory ...
13:
13: lowsFactory.getValidParameters():
13: Solver Type : string = KLU2
13: Refactorization Policy : string = RepivotOnRefactorization
13: Throw on Preconditioner Input : bool = 1
13: VerboseObject ->
13: Verbosity Level : string = default
13: Output File : string = none
13:
13: amesos2LOWSFPL before setting parameters:
13: Solver Type : string = SuperLU_DIST [unused]
13:
13: amesos2LOWSFPL after setting parameters:
13: Solver Type : string = SuperLU_DIST
13: Refactorization Policy : string = RepivotOnRefactorization [default]
13: Throw on Preconditioner Input : bool = 1 [default]
13: VerboseObject ->
13: Output File : string = none [default]
13: Verbosity Level : string = default [default]
13:
13: C) Creating a Amesos2LinearOpWithSolve object nsA from A ...
13: .. Use parMETIS ordering on A'+A with 1 sub-domains.
13: Max szBlk 128
13: Parameters: fill mem 5 fill pelt 5
13: Nonzeros in L 29971
13: Nonzeros in U 19971
13: nonzeros in L+U-I 49942
13: No of supers 9990
13: Size of G(L) 29952
13: Size of G(U) 19962
13: Size of G(L+U) 49914
13: ParSYMBfact (MB) : L\U MAX 0.68 AVG 0.68
13: .. # L blocks 29933 # U blocks 19943
13: MPI tag upper bound = 268435455
13: .. Starting with 1 OpenMP threads
13: === using DAG ===
13: * init: 3.021002e-03 seconds
13: .. thresh = s_eps 5.960464e-08 * anorm 3.999800e+04 = 2.384067e-03
13: .. Buffer size: Lsub 11 Lval 9 Usub 11 Uval 2 LDA 3
13: [0] .. BIG U size 3072
13: [0] .. BIG V size 131072
13: Max row size is 3
13: Using buffer_size of 5000000
13: Threads per process 1
13: Time in scattering 0.000000
13: Time in dgemm 0.000000
13: Total time spent in schur update is : 0.01 seconds,
13: Total Time in Factorization : 0.02 seconds,
13: Time (other GEMM and Scatter) : 0.02 seconds,
13: Total time spent in schur update when offload : 0.00 seconds,
13:
13: D) Testing the LinearOpBase interface of nsA ...
13:
13: *** Entering LinearOpTester<double,double>::check(op,...) ...
13:
13: describe op:
13: Thyra::Amesos2LinearOpWithSolve<double>{rangeDim=10000,domainDim=10000}
13: fwdOp = Thyra::TpetraLinearOp<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> >{rangeDim=10000,domainDim=10000}
13: amesos2Solver=Amesos2::Superludist<Tpetra::CrsMatrix<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace>, false>, Tpetra::MultiVector<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace>, false> >
13:
13: Checking the domain and range spaces ...
13: op.domain().get() != NULL ? passed
13:
13: op.range().get() != NULL ? passed
13:
13: this->check_linear_properties()==true:Checking the linear properties of the forward linear operator ... op.opSupported(NOTRANS) = true == true : passed
13:
13: Checking that the forward operator is truly linear:
13:
13: 0.5*op*(v1 + v2) == 0.5*op*v1 + 0.5*op*v2
13: \_____/ \___/
13: v3 v5
13: \_____________/ \___________________/
13: v4 v5
13:
13: sum(v4) == sum(v5)
13:
13: Random vector tests = 1
13:
13: v1 = randomize(-1,+1); ...
13:
13: v2 = randomize(-1,+1); ...
13:
13: v3 = v1 + v2 ...
13:
13: v4 = 0.5*op*v3 ...
13:
13: v5 = op*v1 ...
13:
13: v5 = 0.5*op*v2 + 0.5*v5 ...
13:
13: Check: rel_err(sum(v4), sum(v5))
13: = rel_err(-0.37757, -0.37757) = 3.23449e-15
13: <= linear_properties_error_tol() = 1e-14 : passed
13: Warning! rel_err(sum(v4), sum(v5))
13: = rel_err(-0.37757, -0.37757) = 3.23449e-15
13: >= linear_properties_warning_tol() = 1e-16!
13:
13: (this->check_linear_properties()&&this->check_adjoint())==false: Skipping the check of the linear properties of the adjoint operator!
13:
13: this->check_adjoint()==false: Skipping check for the agreement of the adjoint and forward operators!
13:
13: this->check_for_symmetry()==false: Skipping check of symmetry ...
13:
13: Congratulations, this LinearOpBase object seems to check out!
13:
13: *** Leaving LinearOpTester<double,double>::check(...)
13:
13: E) Testing the LinearOpWithSolveBase interface of nsA ...
13:
13: *** Entering LinearOpWithSolveTester<double>::check(op,...) ...
13:
13: describe forward op:
13: Thyra::Amesos2LinearOpWithSolve<double>{rangeDim=10000,domainDim=10000}
13: fwdOp = Thyra::TpetraLinearOp<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> >{rangeDim=10000,domainDim=10000}
13: amesos2Solver=Amesos2::Superludist<Tpetra::CrsMatrix<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace>, false>, Tpetra::MultiVector<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace>, false> >
13:
13: this->check_forward_default()==true: Checking the default forward solve ... op.solveSupports(NOTRANS) = true == true : passed
13:
13: Checking that the forward default solve matches the forward operator:
13:
13: inv(Op)*Op*v1 == v1
13: \___/
13: v2
13: \___________/
13: v3
13:
13: v4 = v3-v1
13: v5 = Op*v3-v2
13:
13: norm(v4)/norm(v1) <= forward_default_solution_error_error_tol()
13: norm(v5)/norm(v2) <= forward_default_residual_error_tol()
13:
13: Random vector tests = 1
13:
13: v1 = randomize(-1,+1); ...
13:
13: v2 = Op*v1 ...
13:
13: => Apply time = 8.10623e-05 sec
13:
13: v3 = inv(Op)*v2 ...
13:
13: Solving system using Amesos2 solver Amesos2::Superludist<Tpetra::CrsMatrix<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace>, false>, Tpetra::MultiVector<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace>, false> > ...
13:
13:
13: => Solve time = 0.007236 sec
13:
13: solve status:
13: solveStatus = SOLVE_STATUS_CONVERGED
13: achievedTol = unknownTolerance()
13: message:extraParameters: NONE
13:
13: v4 = v3 - v1 ...
13:
13: v5 = Op*v3 - v2 ...
13:
13: => Apply time = 7.10487e-05 sec
13:
13: Check: |norm(v4)/norm(v1)| = 0.29299 <= forward_default_solution_error_error_tol() = 1e-06 : FAILED
13:
13: Check: |norm(v5)/norm(v2)| = 5.91491e-06 <= forward_default_residual_error_tol() = 2e-06 : FAILED
13:
13: this->check_forward_residual()==true: Checking the forward solve with a tolerance on the residual ... op.solveSupports(NOTRANS) = true == true : passed
13:
13: Checking that the forward solve matches the forward operator to a residual tolerance:
13:
13: v3 = inv(Op)*Op*v1
13: \___/
13: v2
13:
13: v4 = Op*v3-v2
13:
13: norm(v4)/norm(v2) <= forward_residual_solve_tol() + forward_residual_slack_error_tol()
13:
13: Random vector tests = 1
13:
13: v1 = randomize(-1,+1); ...
13:
13: v2 = Op*v1 ...
13:
13: => Apply time = 6.79493e-05 sec
13:
13: v3 = inv(Op)*v2 ...
13:
13: Solving system using Amesos2 solver Amesos2::Superludist<Tpetra::CrsMatrix<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace>, false>, Tpetra::MultiVector<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace>, false> > ...
13:
13:
13: => Solve time = 0.0063262 sec
13:
13: solve status:
13: solveStatus = SOLVE_STATUS_CONVERGED
13: achievedTol = unknownTolerance()
13: message:extraParameters: NONE
13:
13: check: solveStatus = SOLVE_STATUS_CONVERGED == SOLVE_STATUS_CONVERGED : passed
13:
13: v4 = Op*v3 - v2 ...
13:
13: => Apply time = 7.00951e-05 sec
13:
13: Check: |norm(v4)/norm(v2)| = 6.72255e-06 <= forward_residual_solve_tol()+forward_residual_slack_error_tol() = 2e-06 : FAILED
13:
13: this->check_adjoint_default()==false: Skipping the check of the adjoint solve with a default tolerance!
13:
13: this->check_adjoint_residual()==false: Skipping the check of the adjoint solve with a tolerance on the residual!
13:
13: Oh no, at least one of the tests performed with this LinearOpWithSolveBase object failed (see above failures)!
13:
13: *** Leaving LinearOpWithSolveTester<double>::check(...)
13:
13: amesos2LOWSFPL after solving:
13: Solver Type : string = SuperLU_DIST
13: Refactorization Policy : string = RepivotOnRefactorization [default]
13: Throw on Preconditioner Input : bool = 1 [default]
13: VerboseObject ->
13: Output File : string = none [default]
13: Verbosity Level : string = default [default]
13:
13: Oh no! At least one of the tests failed!
1/1 Test #13: Stratimikos_test_single_amesos2_tpetra_solver_driver_SuperLU_DIST_MPI_1 ...***Failed 0.44 sec
0% tests passed, 1 tests failed out of 1
Label Time Summary:
Stratimikos = 0.44 sec (1 test)
Total Test time (real) = 0.48 sec
The following tests FAILED:
13 - Stratimikos_test_single_amesos2_tpetra_solver_driver_SuperLU_DIST_MPI_1 (Failed)
Errors while running CTest
```
It looks like the most outstanding issue is that `inv(A)*A*v != v`, by a large error (~0.2), in part (E) of the testing.
@srajama1 https://gitlab.osti.gov/jmwille/Trilinos/-/issues/1091Set up automated testing with SuperLUDist enabled2017-02-24T16:00:22ZJames WillenbringSet up automated testing with SuperLUDist enabled*Created by: ibaned*
As pointed out in #410, #1083, and #1090, it would be valuable to have automated Trilinos testing that enables the SuperLUDist TPL.
@trilinos/framework *Created by: ibaned*
As pointed out in #410, #1083, and #1090, it would be valuable to have automated Trilinos testing that enables the SuperLUDist TPL.
@trilinos/framework https://gitlab.osti.gov/jmwille/Trilinos/-/issues/1089Stratimikos support for Anasazi2017-02-24T17:40:10ZJames WillenbringStratimikos support for Anasazi*Created by: aprokop*
@trilinos/anasazi @amklinv @hkthorn
If I am not mistaken, Stratimikos currently has not support for Anasazi. If that is correct, how difficult would adding such support be? What are the steps required to do tha...*Created by: aprokop*
@trilinos/anasazi @amklinv @hkthorn
If I am not mistaken, Stratimikos currently has not support for Anasazi. If that is correct, how difficult would adding such support be? What are the steps required to do that? Is there any missing functionality preventing it?
My hope that it may not be hard. The Stratimikos support for MueLu was not hard (and that's my only measuring bar).https://gitlab.osti.gov/jmwille/Trilinos/-/issues/1088Tpetra::MultiVector: Don't use CudaUVMSpace for internal comm buffers2017-08-31T15:06:01ZJames WillenbringTpetra::MultiVector: Don't use CudaUVMSpace for internal comm buffers*Created by: mhoemmen*
@trilinos/tpetra
Blocked by: #1571, #1602, #1658
MPI on Pascal, not K80, appears to be slow when accessing UVM-allocated device buffers, but not standard CUDA device buffers. This causes trouble with Tpetr...*Created by: mhoemmen*
@trilinos/tpetra
Blocked by: #1571, #1602, #1658
MPI on Pascal, not K80, appears to be slow when accessing UVM-allocated device buffers, but not standard CUDA device buffers. This causes trouble with Tpetra::Distributor sending and receiving packed data, as measured by @ambrad . It also probably causes trouble with dot product and norm.
@crtrott and I wrote an MPI + Kokkos benchmark to measure this independently of Tpetra.
Possible work-around: Don't use CudaUVMSpace for internal comm buffers in DistObject methods packAndPrepareNew and unpackAndCombineNew. Define a typedef pack_dev_memory_space which is CudaSpace when execution_space is Cuda.Tpetra-FY17-Q4