Trilinos issueshttps://gitlab.osti.gov/jmwille/Trilinos/-/issues2016-02-05T02:52:08Zhttps://gitlab.osti.gov/jmwille/Trilinos/-/issues/130error: 'isExperimental' was not declared in this scope2016-02-05T02:52:08ZJames Willenbringerror: 'isExperimental' was not declared in this scope*Created by: nschloe*
Commit 12896cfacbb5f051e27e7a4175b6a8c522f3a7f7 introduces the build error
```
packages/ifpack2/src/Ifpack2_RILUK_decl.hpp:673:32: error: 'isExperimental' was not declared in this scope
new_riluk->isExperimenta...*Created by: nschloe*
Commit 12896cfacbb5f051e27e7a4175b6a8c522f3a7f7 introduces the build error
```
packages/ifpack2/src/Ifpack2_RILUK_decl.hpp:673:32: error: 'isExperimental' was not declared in this scope
new_riluk->isExperimental_ = isExperimental;
```
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/83TRILINOS_UNUSED_FUNCTION Doesn't work on windows with intel2016-02-05T02:32:33ZJames WillenbringTRILINOS_UNUSED_FUNCTION Doesn't work on windows with intel*Created by: bmpersc*
This looks like an issue with the intel compiler on windows. It seems that Tpetra has run into this before as the same logic for setting TRILINOS_UNUSED_FUNCTION was checking for windows and not setting TRILINOS_UN...*Created by: bmpersc*
This looks like an issue with the intel compiler on windows. It seems that Tpetra has run into this before as the same logic for setting TRILINOS_UNUSED_FUNCTION was checking for windows and not setting TRILINOS_UNUSED_FUNCTION to **unused** in that case. I have attached a patch that should fix this issue. I have two questions though, is this an acceptable solution? And Why is this logic in 3 places in Trilinos(1 in Teuchos and 2 in Tpetra)?
@mhoemmen
[0001-Teuchos-Fix-compile-issue-on-windows-with-intel.patch.txt](https://github.com/trilinos/Trilinos/files/96309/0001-Teuchos-Fix-compile-issue-on-windows-with-intel.patch.txt)
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/100SEACAS Stray character prevents compilation2016-02-02T21:59:32ZJames WillenbringSEACAS Stray character prevents compilation*Created by: crtrott*
Hi
after the current update of seacas I get the following errors:
```
/home/crtrott/Trilinos/packages/seacas/libraries/suplib/adler.c:1:1: error: expected identifier or \u2018(\u2019 before \u2018/\u2019 token
/h...*Created by: crtrott*
Hi
after the current update of seacas I get the following errors:
```
/home/crtrott/Trilinos/packages/seacas/libraries/suplib/adler.c:1:1: error: expected identifier or \u2018(\u2019 before \u2018/\u2019 token
/home/crtrott/Trilinos/packages/seacas/libraries/suplib/adler.c:8:32: warning: character constant too long for its type [enabled by default]
/home/crtrott/Trilinos/packages/seacas/libraries/suplib/adler.c:16:65: error: unknown type name \u2018you\u2019
/home/crtrott/Trilinos/packages/seacas/libraries/suplib/adler.c:16:74: error: expected \u2018=\u2019, \u2018,\u2019, \u2018;\u2019, \u2018asm\u2019 or \u2018__attribute__\u2019 before \u2018not\u2019
/home/crtrott/Trilinos/packages/seacas/libraries/suplib/adler.c:25:1: error: stray \u2018@\u2019 in program
/home/crtrott/Trilinos/packages/seacas/libraries/suplib/adler.c:25:1: error: stray \u2018@\u2019 in program
In file included from /home/crtrott/Trilinos/packages/seacas/libraries/suplib/adler.c:27:0:
/home/crtrott/Trilinos/packages/seacas/libraries/suplib/adler.h:3:40: error: invalid suffix "AL85000" on integer constant
/home/crtrott/Trilinos/packages/seacas/libraries/suplib/adler.h:27:61: error: unknown type name \u2018LOSS\u2019
/home/crtrott/Trilinos/packages/seacas/libraries/suplib/adler.h:27:69: error: expected \u2018=\u2019, \u2018,\u2019, \u2018;\u2019, \u2018asm\u2019 or \u2018__attribute__\u2019 before \u2018USE\u2019
/home/crtrott/Trilinos/packages/seacas/libraries/suplib/adler.h:28:22: error: unknown type name \u2018OR\u2019
/home/crtrott/Trilinos/packages/seacas/libraries/suplib/adler.h:28:34: error: expected \u2018=\u2019, \u2018,\u2019, \u2018;\u2019, \u2018asm\u2019 or \u2018__attribute__\u2019 before \u2018INTERRUPTION\u2019
/home/crtrott/Trilinos/packages/seacas/libraries/suplib/adler.c:28:41: warning: extra tokens at end of #include directive [enabled by default]
/home/crtrott/Trilinos/packages/seacas/libraries/suplib/adler.c: In function \u2018adler\u2019:
/home/crtrott/Trilinos/packages/seacas/libraries/suplib/adler.c:37:21: warning: declaration of \u2018adler\u2019 shadows a global declaration [-Wshadow]
/home/crtrott/Trilinos/packages/seacas/libraries/suplib/adler.c:37:8: warning: shadowed declaration is here [-Wshadow]
```
Deleting the license header from both
```
/home/crtrott/Trilinos/packages/seacas/libraries/suplib/adler.c
/home/crtrott/Trilinos/packages/seacas/libraries/suplib/adler.h
```
Fixes that, so I guess its just an issue with a windows character or so coming in.
@trilinos/seacas
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/117New "MT Gauss-Seidel" does not seem to work from within MueLu2016-01-30T21:34:28ZJames WillenbringNew "MT Gauss-Seidel" does not seem to work from within MueLu*Created by: aprokop*
@trilinos/ifpack2 @srajama1 @mndevec
In MueLu, we have a driver that allows us to run arbitrary input decks (located in muelu/test/scaling, MueLu_Driver.exe, enabled when MueLu tests are enabled). In particular, ...*Created by: aprokop*
@trilinos/ifpack2 @srajama1 @mndevec
In MueLu, we have a driver that allows us to run arbitrary input decks (located in muelu/test/scaling, MueLu_Driver.exe, enabled when MueLu tests are enabled). In particular, it is possible to test Ifpack2 smoothers with MueLu input deck by constructing a one level hierarchy.
I tried running the driver with the following input deck:
``` xml
<ParameterList name="MueLu">
<Parameter name="max levels" type="int" value="1"/>
<Parameter name="coarse: type" type="string" value="RELAXATION"/>
<ParameterList name="coarse: params">
<Parameter name="relaxation: type" type="string" value="Gauss-Seidel"/>
</ParameterList>
</ParameterList>
```
running it like this `./MueLu_Driver.exe --xml=onelevel-2_np4.xml --solver=standalone`. Here, `standalone` means use MueLu to iterate, so this is essentially a fixed point iteration solver. The result with regular Gauss-Seidel shows expected convergence:
```
iter: 0 residual = {9.99999999999999778e-01}
iter: 1 residual = {2.91355224068349961e-01}
iter: 2 residual = {1.03112694566287258e-01}
iter: 3 residual = {4.24030494030126423e-02}
iter: 4 residual = {2.09351268542225487e-02}
iter: 5 residual = {1.23958077884280798e-02}
iter: 6 residual = {8.39363169782942654e-03}
iter: 7 residual = {6.18796307752396586e-03}
iter: 8 residual = {4.81762942015303165e-03}
iter: 9 residual = {3.89531035831334332e-03}
iter: 10 residual = {3.23934079863641899e-03}
```
I then changed `Gauss-Seidel` in the input deck to `MT Gauss-Seidel` and here is the result:
```
iter: 0 residual = {9.99999999999999778e-01}
iter: 1 residual = {9.99999999999999778e-01}
iter: 2 residual = {9.99999999999999778e-01}
iter: 3 residual = {9.99999999999999778e-01}
iter: 4 residual = {9.99999999999999778e-01}
iter: 5 residual = {9.99999999999999778e-01}
iter: 6 residual = {9.99999999999999778e-01}
iter: 7 residual = {9.99999999999999778e-01}
iter: 8 residual = {9.99999999999999778e-01}
iter: 9 residual = {9.99999999999999778e-01}
iter: 10 residual = {9.99999999999999778e-01}
```
This is definitely not the expected behavior. Even if coloring changes the convergence, it should still converge, right?
I would be happy to work with anybody to fix this issue.
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/51Tpetra::BlockCrsMatrix: Use Kokkos::View<Scalar***> for the "diagonal graph"2016-01-30T10:49:49ZJames WillenbringTpetra::BlockCrsMatrix: Use Kokkos::View<Scalar***> for the "diagonal graph"*Created by: mhoemmen*
@trilinos/tpetra
It would be a lot more efficient to use `Kokkos::View<Scalar***>` for the "diagonal graph," than to use BlockCrsMatrix. We don't need any of that generality; we just need to take a subview of ea...*Created by: mhoemmen*
@trilinos/tpetra
It would be a lot more efficient to use `Kokkos::View<Scalar***>` for the "diagonal graph," than to use BlockCrsMatrix. We don't need any of that generality; we just need to take a subview of each block and apply its inverse. Also, don't store the diagonal graph in BlockCrsMatrix itself; store it in the preconditioner.
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/111Tpetra::Details::FixedHashTable: With CUDA, empty table's hasDuplicateKeys() ...2016-01-30T10:26:57ZJames WillenbringTpetra::Details::FixedHashTable: With CUDA, empty table's hasDuplicateKeys() returns true*Created by: mhoemmen*
@trilinos/tpetra This relates to Tpetra::Details::FixedHashTable. With the CUDA device type only, an empty table's hasDuplicateKeys() returns true. It should return false.
*Created by: mhoemmen*
@trilinos/tpetra This relates to Tpetra::Details::FixedHashTable. With the CUDA device type only, an empty table's hasDuplicateKeys() returns true. It should return false.
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/105Tpetra::Details::FixedHashTable: copyOffsets does incorrect host access from ...2016-01-28T13:44:04ZJames WillenbringTpetra::Details::FixedHashTable: copyOffsets does incorrect host access from CUDA*Created by: mhoemmen*
Tpetra::Details::FixedHashTable: Fix CUDA crash
@trilinos/tpetra
FixedHashTable implements Tpetra::Map's conversion from global to local
indices for the noncontiguous Map case, among other things. It has a
"cop...*Created by: mhoemmen*
Tpetra::Details::FixedHashTable: Fix CUDA crash
@trilinos/tpetra
FixedHashTable implements Tpetra::Map's conversion from global to local
indices for the noncontiguous Map case, among other things. It has a
"copy constructor" which converts between instances on different Kokkos
devices. This constructor uses the copyOffsets internal function, which
lives in Tpetra_Details_FixedHashTable_decl.hpp. copyOffsets copies
FixedHashTable::ptr_ -- the offsets array (corresponds to the 'ptr'
array in CSR) -- from one device to another. Different devices may use
different offset types, so copyOffsets can't just use Kokkos::deep_copy.
It also checks for overflow. Thus, it uses a custom parallel_reduce
functor, CopyOffsets (note initial capital).
The CopyOffsets functor is templated on the input and output View types.
It assumes that the output View's execution space can access the input
View's memory space. This is a bug. For example, if the output View's
execution space is Kokkos::Cuda, it cannot access host memory
(Kokkos::HostSpace). (That's the wrong direction for CUDA UVM.) This
is why the Dashboard was showing a failed test: the test was throwing in
this case, because Kokkos was catching the incorrect access at run time.
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/104Tpetra::Vector::normWeighted emits a deprecated warning2016-01-28T08:20:41ZJames WillenbringTpetra::Vector::normWeighted emits a deprecated warning*Created by: mhoemmen*
@trilinos/tpetra
Tpetra::Vector::normWeighted is deprecated. There's nothing wrong with deprecation. However, the method calls its parent class' method MultiVector::normWeighted, which is also deprecated. Eve...*Created by: mhoemmen*
@trilinos/tpetra
Tpetra::Vector::normWeighted is deprecated. There's nothing wrong with deprecation. However, the method calls its parent class' method MultiVector::normWeighted, which is also deprecated. Even though nothing calls Vector::normWeighted any more, some compilers still emit a deprecated warning when building it, because it calls a deprecated method. This breaks builds that turn on warnings as errors.
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/90Segfault in Tpetra's writeDenseFile()2016-01-24T13:32:40ZJames WillenbringSegfault in Tpetra's writeDenseFile()*Created by: dridzal*
@trilinos/tpetra: The @trilinos/rol team is using MatrixMarket writers for the PDE-OPT application development kit. It appears that a recent change to Tpetra is causing segaults in writeDenseFile(), in an MPI bui...*Created by: dridzal*
@trilinos/tpetra: The @trilinos/rol team is using MatrixMarket writers for the PDE-OPT application development kit. It appears that a recent change to Tpetra is causing segaults in writeDenseFile(), in an MPI build, for example in the data.hpp file in /rol/example/PDE-OPT/poisson:
void outputTpetraVector(const Teuchos::RCP<const Tpetra::MultiVector<> > &vec,
const std::string &filename) const {
Tpetra::MatrixMarket::WriterTpetra::MultiVector< > vecWriter;
vecWriter.writeDenseFile(filename, vec);
}
We are not sure which recent change in Tpetra caused the issue; it was a day or two after the ROL commit d5bcec2467f9087f0117ae7da1db921b3afc4d6d. If you checkout this sha1, the example in the above directory will run without segfaults, in a parallel MPI build. The issue is not present in a serial build. We have not checked CrsMatrix writers.
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/84Xpetra::MultiVector tests segfaulting after recent changes in Tpetra2016-01-23T03:18:19ZJames WillenbringXpetra::MultiVector tests segfaulting after recent changes in Tpetra*Created by: tawiesn*
The Tpetra stack tests in Xpetra for the MultiVector are segfaulting after the recent changes in the Tpetra::MultiVector:
==29656== at 0x989A7C8: Tpetra::MultiVector<double, int, long long, Kokkos::Compat::Kokk...*Created by: tawiesn*
The Tpetra stack tests in Xpetra for the MultiVector are segfaulting after the recent changes in the Tpetra::MultiVector:
==29656== at 0x989A7C8: Tpetra::MultiVector<double, int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::OpenMP, Kokkos::HostSpace>, false>::MultiVector(Teuchos::RCP<Tpetra::Map<int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::OpenMP, Kokkos::HostSpace> > const> const&, Kokkos::DualView<double**, Kokkos::LayoutLeft, Kokkos::OpenMP, void> const&, Kokkos::DualView<double**, Kokkos::LayoutLeft, Kokkos::OpenMP, void> const&) (Tpetra_MultiVector_def.hpp:403)
==29656== Address 0x29d is not stack'd, malloc'd or (recently) free'd
The problem seems to result from one of the changes between the commits a244ea7cf43 and 369d2c3d17da.
In my configuration ETI=on
@trilinos/tpetra
@trilinos/xpetra
@mhoemmen
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/92Kokkos: invalid use of incomplete type build error on CEE linux2016-01-23T02:18:30ZJames WillenbringKokkos: invalid use of incomplete type build error on CEE linux*Created by: dicengine*
This has been an issue for the last couple days with building VOTD Trilinos. Here are the configure results, build error, and cmake configuration files.
[make.txt](https://github.com/trilinos/Trilinos/files/1007...*Created by: dicengine*
This has been an issue for the last couple days with building VOTD Trilinos. Here are the configure results, build error, and cmake configuration files.
[make.txt](https://github.com/trilinos/Trilinos/files/100780/make.txt)
[cmake.txt](https://github.com/trilinos/Trilinos/files/100781/cmake.txt)
[do-cmake.txt](https://github.com/trilinos/Trilinos/files/100782/do-cmake.txt)
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/50Tpetra::BlockCrsMatrix: Add LU factorization, solve, & inverse for small dens...2016-01-19T22:44:44ZJames WillenbringTpetra::BlockCrsMatrix: Add LU factorization, solve, & inverse for small dense blocks*Created by: mhoemmen*
@trilinos/tpetra
Add GETRF (LU factorization), GETRS (solve linear system(s) using results of GETRF), and GETRI (compute explicit matrix inverse, using results of GETRF) for LittleBlock in BlockCrsMatrix.
GETRI ...*Created by: mhoemmen*
@trilinos/tpetra
Add GETRF (LU factorization), GETRS (solve linear system(s) using results of GETRF), and GETRI (compute explicit matrix inverse, using results of GETRF) for LittleBlock in BlockCrsMatrix.
GETRI will help accelerate block Jacobi (a.k.a. point implicit), by pushing solve costs to setup and making computation during the actual apply step just little dense matrix-vector multiplies. This should be easier to vectorize.
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/34Xpetra: many tests fail when Epetra, Tpetra, and OpenMP enabled2016-01-18T21:53:14ZJames WillenbringXpetra: many tests fail when Epetra, Tpetra, and OpenMP enabled*Created by: mhoemmen*
@trilinos/xpetra
When Epetra is enabled, Xpetra uses `Kokkos::Compat::KokkosSerialWrapperNode` as its Node type. (This corresponds to the `Kokkos::Serial` execution space.) There would be nothing particularly ...*Created by: mhoemmen*
@trilinos/xpetra
When Epetra is enabled, Xpetra uses `Kokkos::Compat::KokkosSerialWrapperNode` as its Node type. (This corresponds to the `Kokkos::Serial` execution space.) There would be nothing particularly wrong with that, except that if Tpetra is also enabled, Xpetra then tries to instantiate Tpetra for that Node type. This is a problem because Tpetra does not necessarily enable that Node type. For example, if OpenMP is enabled (`Trilinos_ENABLE_OpenMP=ON`), Tpetra only enables the OpenMP Node type (`Kokkos::Compat::KokkosOpenMPWrapperNode`) by default.
My recent commit fixed the MPI_DEBUG build due to the above issue:
https://github.com/trilinos/Trilinos/commit/64c7185b0b60411f65757ad3abc1c333470bdf83
but it does NOT fix several failing tests. My guess is that the test failures are related.
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/76Get Trilinos to adopt the XSDK configure standards2016-01-14T16:28:03ZJames WillenbringGet Trilinos to adopt the XSDK configure standards*Created by: bartlettroscoe*
Trilinos needs to adopt the updated xSDK configure options (see [SDK-1](https://ideas-productivity.atlassian.net/browse/SDK-1) and [SDK-2](https://ideas-productivity.atlassian.net/browse/SDK-2)). I have thi...*Created by: bartlettroscoe*
Trilinos needs to adopt the updated xSDK configure options (see [SDK-1](https://ideas-productivity.atlassian.net/browse/SDK-1) and [SDK-2](https://ideas-productivity.atlassian.net/browse/SDK-2)). I have this done and integrated into TriBITS (see [TriBITS #107](https://github.com/TriBITSPub/TriBITS/issues/107)) but it needs some review. @jwillenbring, given that you are also funded on the IDEAS project, do you think you can give [TriBITS #107](https://github.com/TriBITSPub/TriBITS/issues/107) a review? You can also manually test that it is behaving according to the [xSDK configure standard](https://docs.google.com/document/d/18028D6nsuhIrCvJnX6c07r8m_Np4SH-aGXMX4svMs1w).
Anyway, I am about to snapshot the updated TriBITS to the Trilinos 'master' branch.
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/77Amesos_Mumps.cpp does not build when MPI is disabled2016-01-13T15:29:45ZJames WillenbringAmesos_Mumps.cpp does not build when MPI is disabled*Created by: bavier*
Configuring with cmake 3.3.2 at master@e9a607a with
```
$ mkdir build && cd build
$ cmake \
-DTrilinos_ENABLE_Amesos:BOOL=ON \
-DTPL_ENABLE_MUMPS:BOOL=ON \
..
```
then running `make` produces the follo...*Created by: bavier*
Configuring with cmake 3.3.2 at master@e9a607a with
```
$ mkdir build && cd build
$ cmake \
-DTrilinos_ENABLE_Amesos:BOOL=ON \
-DTPL_ENABLE_MUMPS:BOOL=ON \
..
```
then running `make` produces the following error:
```
/ptmp/trilinos-12.6.1-rc/packages/amesos/src/Amesos_Mumps.cpp: In member function 'virtual int Amesos_Mumps::SymbolicFactorization()':
/ptmp/trilinos-12.6.1-rc/packages/amesos/src/Amesos_Mumps.cpp:478:9: error: 'Epetra_MpiComm' does not name a type
const Epetra_MpiComm* MpiComm = dynamic_cast<const Epetra_MpiComm*>(&Comm());
^
In file included from /gnu/store/gma0ig9v61y7c1rhvaj7qx1ga86wnjs7-gfortran-4.9.3/include/c++/cassert:43:0,
from /ptmp/trilinos-12.6.1-rc/packages/teuchos/core/src/Teuchos_ConfigDefs.hpp:93,
from /ptmp/trilinos-12.6.1-rc/packages/teuchos/core/src/Teuchos_RCPNode.hpp:52,
from /ptmp/trilinos-12.6.1-rc/packages/teuchos/core/src/Teuchos_RCPDecl.hpp:51,
from /ptmp/trilinos-12.6.1-rc/packages/teuchos/core/src/Teuchos_RCP.hpp:58,
from /ptmp/trilinos-12.6.1-rc/packages/amesos/src/Amesos_BaseSolver.h:48,
from /ptmp/trilinos-12.6.1-rc/packages/amesos/src/Amesos_Mumps.h:44,
from /ptmp/trilinos-12.6.1-rc/packages/amesos/src/Amesos_Mumps.cpp:29:
/ptmp/trilinos-12.6.1-rc/packages/amesos/src/Amesos_Mumps.cpp:479:11: error: 'MpiComm' was not declared in this scope
assert (MpiComm != 0);
^
/ptmp/trilinos-12.6.1-rc/packages/amesos/src/Amesos_Mumps.cpp:480:68: error: 'MPI_Comm_c2f' was not declared in this scope
MDS.comm_fortran = (MUMPS_INT) MPI_Comm_c2f(MpiComm->GetMpiComm());
^
```
I would expect configuration to fail, rather than the build to fail, if Amesos_Mumps is not supported when MPI is disabled. On the other hand, MUMPS supports seriail execution, so perhaps it would be best to check for MUMPS's libmpiseq during configuration if TPL_ENABLE_MPI:BOOL=OFF and then adjust Amesos_Mumps accordingly.
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/68describe with extreme verbosity only shows info on process 02016-01-07T00:49:11ZJames Willenbringdescribe with extreme verbosity only shows info on process 0*Created by: nschloe*
The output of
``` c++
#include <Teuchos_DefaultComm.hpp>
#include <Teuchos_VerboseObject.hpp>
#include <Tpetra_Map.hpp>
int main ( int argc, char *argv[] )
{
Teuchos::GlobalMPISession session(&argc, &argv, NULL...*Created by: nschloe*
The output of
``` c++
#include <Teuchos_DefaultComm.hpp>
#include <Teuchos_VerboseObject.hpp>
#include <Tpetra_Map.hpp>
int main ( int argc, char *argv[] )
{
Teuchos::GlobalMPISession session(&argc, &argv, NULL);
auto comm = Teuchos::DefaultComm<int>::getComm();
Tpetra::Map<int,int> map(23, 0, comm);
auto out = Teuchos::VerboseObjectBase::getDefaultOStream();
map.describe(*out, Teuchos::VERB_EXTREME);
return EXIT_SUCCESS;
}
```
when run on two procs is
```
$ make && mpiexec -n 2 ./tpetra-test
Process 0:
My number of entries: 12
My minimum global index: 0
My maximum global index: 11
My global indices: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]
```
The same goes for vectors, probably other types.
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/65unexpected (and silent) sumIntoGlobalValues failure2016-01-04T16:00:47ZJames Willenbringunexpected (and silent) sumIntoGlobalValues failure*Created by: nschloe*
@trilinos/tpetra
I have a larger application code in which I build up a matrix by looping over the edges of a mesh and sum values into the matrix for each of the two end points. I found that some indices aren't co...*Created by: nschloe*
@trilinos/tpetra
I have a larger application code in which I build up a matrix by looping over the edges of a mesh and sum values into the matrix for each of the two end points. I found that some indices aren't correctly inserted when running on more than one processes.
I could boil it down to
``` c++
#include <Teuchos_DefaultComm.hpp>
#include <Tpetra_CrsMatrix.hpp>
int main ( int argc, char *argv[] )
{
Teuchos::GlobalMPISession session(&argc, &argv, NULL);
// Initialize MPI
auto comm = Teuchos::DefaultComm<int>::getComm();
// build map in which process K owns row K
std::vector<int> ids = {comm->getRank()};
auto map = Teuchos::rcp(new Tpetra::Map<int>(-1, ids, 0, comm));
const int globalSize = map->getGlobalNumElements();
// insert indices from all processes
auto graph = Teuchos::rcp(new Tpetra::CrsGraph<int, int>(map, map, 0));
for (int i = 0; i < globalSize; i++) {
for (int j = 0; j < globalSize; j++) {
graph->insertGlobalIndices(i, Teuchos::tuple(j));
}
}
graph->fillComplete();
// fill matrix
auto A = Teuchos::rcp(new Tpetra::CrsMatrix<double,int,int>(graph));
A->setAllToScalar(0.0);
for (size_t k = 0; k < globalSize; k++) {
const int num = A->sumIntoGlobalValues(
k,
Teuchos::tuple(0, 1),
Teuchos::tuple(0.33, 1.55)
);
std::cout << "added from @" << comm->getRank() << " in row " << k << ": "
<< num
<< std::endl;
}
A->fillComplete();
return EXIT_SUCCESS;
}
```
When run on two procs, I'm getting
```
$ mpiexec -n 2 ./test
added from @0 in row 0: 1
added from @0 in row 1: 2
added from @1 in row 0: 2
added from @1 in row 1: 1
```
I would have expected that two entries are inserted for each `sumIntoGlobalValues`. The `insertGlobalIndices` I thought makes sure that the corresponding "slots" are present in the respective processes. Apparently, that's not the case.
When checking out the actual values after `A->fillComplete()` with
``` c++
Teuchos::Array<int> cols(100);
Teuchos::Array<double> vals(100);
size_t numss;
for (size_t k = 0; k < globalSize; k++) {
A->getGlobalRowCopy(k, cols, vals, numss);
std::cout << "@" << comm->getRank()
<< ", row " << k
<< ", num columns " << numss
<< ", columns ";
for (size_t kk = 0; kk < numss; kk++) {
std::cout << cols[kk] << " ";
}
std::cout << ", values ";
for (size_t kk = 0; kk < numss; kk++) {
std::cout << vals[kk] << " ";
}
std::cout << std::endl;
}
```
I'm getting
```
@0, row 0, num columns 2, columns -1 0 , values 0 0.66
@0, row 1, num columns 0, columns , values
@1, row 0, num columns 0, columns , values
@1, row 1, num columns 2, columns -1 1 , values 0 3.1
```
On a side node, is there a way to make Tpetra throw when this happens? Having to check the return value to assert correct operation seems a little Epetry.
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/63Tpetra::Distributor: Inheriting from VerboseObject<Distributor> causes crash ...2015-12-24T23:20:42ZJames WillenbringTpetra::Distributor: Inheriting from VerboseObject<Distributor> causes crash in Import ctor w/ various Intel compiler versions*Created by: mhoemmen*
@trilinos/tpetra @amklinv
With Intel 15.0.4 and Intel 16, using either Intel MPI or OpenMPI 1.8, the Tpetra::Import constructor crashes. Micah chased this down and found out that the Tpetra::Distributor instanc...*Created by: mhoemmen*
@trilinos/tpetra @amklinv
With Intel 15.0.4 and Intel 16, using either Intel MPI or OpenMPI 1.8, the Tpetra::Import constructor crashes. Micah chased this down and found out that the Tpetra::Distributor instance that Import creates inherits from Teuchos::VerboseObjectTpetra::Distributor (looks weird, I know, but this is how one is supposed to use VerboseObject), and that VerboseObject's constructor was getting into a weird multiply segfaulting infinite loop when calling initializeVerboseObjectBase().
Some versions of the Intel compiler have trouble with name resolution in certain cases. On a hunch, I made Distributor no longer inherit from VerboseObject, and that fixed the problem. Yay!
Thus, the fix is to remove Distributor's inheritance from VerboseObject. This was pretty easy to do but I want to make sure that Distributor's ParameterList still accepts a "VerboseObject" sublist (see e.g., line 430 of Tpetra_Distributor.cpp).
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/60build failure: multiple definition of `void Xpetra::Jacobi2015-12-18T20:54:06ZJames Willenbringbuild failure: multiple definition of `void Xpetra::Jacobi*Created by: nschloe*
The current master fails to build with
```
[...]
Wl,-Bdynamic ../../kokkos/algorithms/src/libtrilinos_kokkosalgorithms.so.12.5 ../../kokkos/core/src/libtrilinos_kokkoscore.so.12.5
CMakeFiles/trilinos_stokhos_muel...*Created by: nschloe*
The current master fails to build with
```
[...]
Wl,-Bdynamic ../../kokkos/algorithms/src/libtrilinos_kokkosalgorithms.so.12.5 ../../kokkos/core/src/libtrilinos_kokkoscore.so.12.5
CMakeFiles/trilinos_stokhos_muelu.dir/sacado/kokkos/pce/muelu/MueLu_SaPFactory_UQ_PCE_Serial.cpp.o: In function `Xpetra::TpetraCrsGraph<int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> >::getMap() const':
/«PKGBUILDDIR»/packages/stokhos/src/sacado/kokkos/pce/Sacado_UQ_PCE.hpp:145: multiple definition of `void Xpetra::Jacobi<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> >(double, Xpetra::Vector<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> > const&, Xpetra::Matrix<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> > const&, Xpetra::Matrix<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> > const&, Xpetra::Matrix<double, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> >&, bool, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
```
Full details (including configuration) [at Launchpad](https://launchpadlibrarian.net/230424356/buildlog_ubuntu-xenial-amd64.trilinos_12.5~20151218030407-xenial1_BUILDING.txt.gz).
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/54Tpetra,Ifpack2: Add GEMM for small dense blocks2015-12-18T17:39:26ZJames WillenbringTpetra,Ifpack2: Add GEMM for small dense blocks*Created by: mhoemmen*
@trilinos/tpetra @amklinv @jhux2 @csiefer2
Ifpack2's incomplete LU factorization wants a matrix-matrix multiply (GEMM, in BLAS terms) for small dense matrices (the "blocks" in Tpetra's BlockCrsMatrix). The exis...*Created by: mhoemmen*
@trilinos/tpetra @amklinv @jhux2 @csiefer2
Ifpack2's incomplete LU factorization wants a matrix-matrix multiply (GEMM, in BLAS terms) for small dense matrices (the "blocks" in Tpetra's BlockCrsMatrix). The existing implementation (at the top of Ifpack2_Experimental_RBILUK_decl.hpp) assumes a particular layout of blocks.
This is like Issue #50, in that the eventual goal is to make all of these computational kernels available in KokkosKernels. We're starting by putting the kernels in the Tpetra::Experimental namespace.