Trilinos issueshttps://gitlab.osti.gov/jmwille/Trilinos/-/issues2019-06-08T15:27:26Zhttps://gitlab.osti.gov/jmwille/Trilinos/-/issues/5098Xpetra fatal error due to missing headers, similar to #50112019-06-08T15:27:26ZJames WillenbringXpetra fatal error due to missing headers, similar to #5011*Created by: spdomin*
We have another missing Xpetra missing header file:
In file included from /scratch/spdomin/nightlyBuildAndTest/NaluNightly/include/LinearSolver.h:34:0,
from /scratch/spdomin/nightlyBuildAndTest...*Created by: spdomin*
We have another missing Xpetra missing header file:
In file included from /scratch/spdomin/nightlyBuildAndTest/NaluNightly/include/LinearSolver.h:34:0,
from /scratch/spdomin/nightlyBuildAndTest/NaluNightly/src/EnthalpyEquationSystem.C:43:
/home/spdomin/gitHubWork/scratch_build/install/gcc7.2.0/Trilinos_nightly_release/include/MueLu.hpp:63:10: fatal error: Xpetra_CrsMatrixWrap.hpp: No such file or directory
#include <Xpetra_CrsMatrixWrap.hpp>
This looks almost the same as the foamier issue that @jhux2 fixed and what @bartlettroscoe asked, "How did this pass PR testing? Is the file Xpetra_CrsMatrixFactory.hpp not included in any automated tests run in PR testing? Or are globs not used to install header files."
https://github.com/trilinos/Trilinos/issues/5011
Best,https://gitlab.osti.gov/jmwille/Trilinos/-/issues/4038Xpetra/Epetra: link error in panzer test for intel debug static2019-04-26T16:35:24ZJames WillenbringXpetra/Epetra: link error in panzer test for intel debug static*Created by: bathmatt*
<!---
Provide a general summary of the issue in the Title above. If this issue
pertains to a particular package in Trilinos, it's worthwhile to start the
title with "PackageName: ".
-->
When I try to link p...*Created by: bathmatt*
<!---
Provide a general summary of the issue in the Title above. If this issue
pertains to a particular package in Trilinos, it's worthwhile to start the
title with "PackageName: ".
-->
When I try to link packages/panzer/adapters-stk/test/solver/PanzerAdaptersSTK_solver.exe I get a missing tpetra symbol for intel debug static, but intel opt static works.. No idea why. This is with a RHEL7 machine using all sems and Ross' new build scripts.
<!---
Note that anything between these delimiters is a comment that will not appear
in the issue description once created. Click on the Preview tab to see what
everything will look like when you submit.
-->
Here is the error message I see, no idea why the xpetra epetra map is looking for something tpetra??
```
packages/xpetra/src/libxpetra.a(Xpetra_EpetraMap.cpp.o): In function `Tpetra::Details::LocalMap<int, long, Kokkos::Device<Kokkos::OpenMP, Kokkos::HostSpace> >::LocalMap()':
```
Here is my link line
```
: && /projects/sems/install/rhel7-x86_64/sems/compiler/intel/17.0.1/openmpi/1.10.1/bin/mpicxx --std=c++11 -g -fopenmp -g packages/panzer/adapters-stk/test/solver/CMakeFiles/PanzerAdaptersSTK_solver.dir/solver.cpp.o packages/panzer/adapters-stk/test/solver/CMakeFiles/PanzerAdaptersSTK_solver.dir/__/__/__/__/phalanx/test/Utilities/Phalanx_UnitTestMain.cpp.o -o packages/panzer/adapters-stk/test/solver/PanzerAdaptersSTK_solver.exe packages/panzer/adapters-stk/src/libpanzer-stk.a packages/muelu/adapters/libmuelu-adapters.a packages/muelu/src/Interface/libmuelu-interface.a packages/muelu/src/libmuelu.a packages/teko/src/libteko.a packages/ifpack2/adapters/libifpack2-adapters.a packages/ifpack2/src/libifpack2.a packages/seacas/libraries/ioss/src/main/libio_info_lib.a packages/seacas/libraries/ioss/src/init/libIonit.a packages/seacas/libraries/ioss/src/transform/libIotr.a packages/seacas/libraries/ioss/src/heartbeat/libIohb.a packages/seacas/libraries/ioss/src/gen_struc/libIogs.a packages/seacas/libraries/ioss/src/generated/libIogn.a packages/seacas/libraries/ioss/src/visualization/libIovs.a packages/seacas/libraries/ioss/src/pamgen/libIopg.a packages/seacas/libraries/ioss/src/exo_fac/libIoexo_fac.a packages/seacas/libraries/ioss/src/exo_fpp/libIofx.a packages/seacas/libraries/ioss/src/exodus/libIoex.a packages/seacas/libraries/ioss/src/libIoss.a packages/seacas/libraries/exodus/libexodus.a packages/panzer/disc-fe/src/libpanzer-disc-fe.a packages/panzer/dof-mgr/src/libpanzer-dof-mgr.a packages/phalanx/src/libphalanx.a packages/panzer/core/src/libpanzer-core.a packages/piro/src/libpiro.a packages/muelu/adapters/libmuelu-adapters.a packages/muelu/src/Interface/libmuelu-interface.a packages/muelu/src/libmuelu.a packages/intrepid2/src/libintrepid2.a packages/sacado/src/libsacado.a packages/tempus/src/libtempus.a packages/rythmos/src/librythmos.a packages/nox/src-loca/src-thyra/liblocathyra.a packages/nox/src-loca/src-epetra/liblocaepetra.a packages/nox/src-loca/src-lapack/liblocalapack.a packages/nox/src-loca/src/libloca.a packages/nox/src-epetra/libnoxepetra.a packages/nox/src-lapack/libnoxlapack.a packages/nox/src/libnox.a packages/teko/src/libteko.a packages/ifpack2/adapters/libifpack2-adapters.a packages/ifpack2/src/libifpack2.a packages/zoltan2/src/libzoltan2.a packages/anasazi/tpetra/src/libanasazitpetra.a packages/anasazi/epetra/util/ModeLaplace/libModeLaplace.a packages/anasazi/epetra/src/libanasaziepetra.a packages/anasazi/src/libanasazi.a packages/stk/stk_io/stk_io/util/libstk_io_util.a packages/stk/stk_io/stk_io/libstk_io.a packages/seacas/libraries/ioss/src/main/libio_info_lib.a packages/seacas/libraries/ioss/src/init/libIonit.a packages/seacas/libraries/ioss/src/transform/libIotr.a packages/seacas/libraries/ioss/src/heartbeat/libIohb.a packages/seacas/libraries/ioss/src/gen_struc/libIogs.a packages/seacas/libraries/ioss/src/generated/libIogn.a packages/seacas/libraries/ioss/src/visualization/libIovs.a packages/seacas/libraries/ioss/src/pamgen/libIopg.a packages/seacas/libraries/ioss/src/exo_fac/libIoexo_fac.a packages/seacas/libraries/ioss/src/exo_fpp/libIofx.a packages/seacas/libraries/ioss/src/exodus/libIoex.a packages/seacas/libraries/ioss/src/libIoss.a packages/pamgen/src/libpamgen_extras.a packages/pamgen/src/libpamgen.a packages/stk/stk_mesh/stk_mesh/base/libstk_mesh_base.a packages/shards/src/libshards.a packages/stk/stk_topology/stk_topology/libstk_topology.a packages/stk/stk_util/stk_util/use_cases/libstk_util_use_cases.a packages/stk/stk_util/stk_util/registry/libstk_util_registry.a packages/stk/stk_util/stk_util/diag/libstk_util_diag.a packages/stk/stk_util/stk_util/environment/libstk_util_env.a packages/stk/stk_util/stk_util/parallel/libstk_util_parallel.a packages/stk/stk_util/stk_util/util/libstk_util_util.a /projects/sems/install/rhel7-x86_64/sems/tpl/boost/1.59.0/intel/17.0.1/base/lib/libboost_program_options.so /projects/sems/install/rhel7-x86_64/sems/tpl/boost/1.59.0/intel/17.0.1/base/lib/libboost_system.so packages/seacas/libraries/aprepro_lib/libaprepro_lib.a packages/seacas/libraries/exodus/libexodus.a -L/projects/sems/install/rhel7-x86_64/sems/tpl/hdf5/1.8.12/intel/17.0.1/openmpi/1.10.1/parallel/lib -L/projects/sems/install/rhel7-x86_64/sems/tpl/boost/1.59.0/intel/17.0.1/base/lib -L/projects/sems/install/rhel7-x86_64/sems/tpl/netcdf/4.4.1/intel/17.0.1/openmpi/1.10.1/exo_parallel/lib -L/lib -lboost_program_options -lboost_system -lnetcdf -lpnetcdf -lhdf5 -lcurl -lhdf5_hl -lz packages/stratimikos/src/libstratimikos.a packages/stratimikos/adapters/belos/src/libstratimikosbelos.a packages/stratimikos/adapters/amesos2/src/libstratimikosamesos2.a packages/stratimikos/adapters/aztecoo/src/libstratimikosaztecoo.a packages/stratimikos/adapters/amesos/src/libstratimikosamesos.a packages/stratimikos/adapters/ml/src/libstratimikosml.a packages/stratimikos/adapters/ifpack/src/libstratimikosifpack.a packages/amesos2/src/libamesos2.a packages/ml/src/libml.a packages/galeri/src-xpetra/libgaleri-xpetra.a packages/galeri/src-epetra/libgaleri-epetra.a packages/ifpack/src/libifpack.a packages/amesos/src/libamesos.a packages/common/auxiliarySoftware/SuiteSparse/src/libtrilinosss.a packages/belos/xpetra/src/libbelosxpetra.a packages/belos/tpetra/src/libbelostpetra.a packages/belos/epetra/src/libbelosepetra.a packages/belos/src/libbelos.a packages/xpetra/sup/libxpetra-sup.a packages/xpetra/src/libxpetra.a packages/thyra/adapters/tpetra/src/libthyratpetra.a packages/thyra/adapters/epetraext/src/libthyraepetraext.a packages/epetraext/src/libepetraext.a packages/thyra/adapters/epetra/src/libthyraepetra.a packages/thyra/core/src/libthyracore.a packages/rtop/src/librtop.a packages/tpetra/core/ext/libtpetraext.a packages/tpetra/core/inout/libtpetrainout.a packages/tpetra/core/src/libtpetra.a packages/kokkos-kernels/src/libkokkoskernels.a packages/kokkos/algorithms/src/libkokkosalgorithms.a packages/kokkos/containers/src/libkokkoscontainers.a packages/tpetra/classic/LinAlg/libtpetraclassiclinalg.a packages/tpetra/classic/NodeAPI/libtpetraclassicnodeapi.a packages/tpetra/classic/src/libtpetraclassic.a packages/aztecoo/src/libaztecoo.a packages/triutils/src/libtriutils.a packages/epetra/src/libepetra.a packages/teuchos/kokkoscomm/src/libteuchoskokkoscomm.a packages/teuchos/kokkoscompat/src/libteuchoskokkoscompat.a packages/teuchos/remainder/src/libteuchosremainder.a packages/teuchos/numerics/src/libteuchosnumerics.a packages/teuchos/comm/src/libteuchoscomm.a packages/teuchos/parameterlist/src/libteuchosparameterlist.a packages/teuchos/parser/src/libteuchosparser.a -mkl packages/teuchos/core/src/libteuchoscore.a packages/kokkos/core/src/libkokkoscore.a -ldl packages/zoltan/src/libzoltan.a -lm && :
```
<!---
Feel free to delete anything from this template that is not applicable to the
issue you are submitting.
-->
<!---
Replace <teamName> below with the appropriate Trilinos package/team name.
-->
@trilinos/<teamName>
<!---
Assignees: If you know anyone who should likely tackle this issue, select them
from the Assignees drop-down on the right.
-->
<!---
Lables: Choose any applicable package names from the Labels drop-down on the
right. Additionally, choose a label to indicate the type of issue, for
instance, bug, build, documentation, enhancement, etc.
-->
## Expectations
<!---
Tell us what you think should happen, how you think things should work, what
you would like to see in the documentation, etc.
-->
## Current Behavior
<!---
Tell us how the current behavior fails to meet your expectations in some way.
-->
## Motivation and Context
<!---
How has this expectation failure affected you? What are you trying to
accomplish? Why do we need to address this? What does it have to do with
anything? Providing context helps us come up with a solution that is most
useful in the real world.
-->
## Definition of Done
<!---
Tell us what needs to happen. If necessary, give us a task list along the
lines of:
- [ ] First do this.
- [ ] Then do that.
- [ ] Also this other thing.
-->
## Possible Solution
<!---
Not obligatory, but suggest a fix for the bug or documentation, or suggest
ideas on how to implement the addition or change.
-->
## Steps to Reproduce
<!---
Provide a link to a live example, or an unambiguous set of steps to reproduce
this issue. Include code to reproduce, if relevant.
1. Do this.
1. Do that.
1. Shake fist angrily at computer.
-->
## Your Environment
<!---
Include relevant details about your environment such that we can replicate this
issue.
-->
- **Relevant repo SHA1s:**
- **Relevant configure flags or configure script:**
- **Operating system and version:**
- **Compiler and TPL versions:**
## Related Issues
<!---
If applicable, let us know how this bug is related to any other open issues:
-->
* Blocks
* Is blocked by
* Follows
* Precedes
* Related to
* Part of
* Composed of
## Additional Information
<!---
Anything else that might be helpful for us to know in addressing this issue:
* Configure log file:
* Build log file:
* Test log file:
* When was the last time everything worked (date/time; SHA1s; etc.)?
* What did you do that made the bug rear its ugly head?
* Have you tried turning it off and on again?
-->
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/3939PanzerAdaptersSTK_[Mixed]CurlLaplacianExample tests failing in Trilinos-atdm-...2018-12-22T19:45:49ZJames WillenbringPanzerAdaptersSTK_[Mixed]CurlLaplacianExample tests failing in Trilinos-atdm-waterman-cuda-9.2-release-debug build*Created by: fryeguy52*
CC: @trilinos/panzer, @mperego (Trilinos Discretizations Product Lead), @bartlettroscoe, @fryeguy52
## Next Action Status
With the merge of PR #4079 to 'develop' on 12/19/2018, these tests should now be dis...*Created by: fryeguy52*
CC: @trilinos/panzer, @mperego (Trilinos Discretizations Product Lead), @bartlettroscoe, @fryeguy52
## Next Action Status
With the merge of PR #4079 to 'develop' on 12/19/2018, these tests should now be disabled in this build `Trilinos-atdm-waterman-cuda-9.2-release-debug`. All tests that should be disabled were disabled on 12/19/2018 and all of the Panzer tests in this build passed on 12/19/2018 and 12/20/2018.
## Description
As shown in [this query](https://testing.sandia.gov/cdash/queryTests.php?project=Trilinos&filtercombine=and&filtercombine=&filtercombine=and&filtercombine=and&filtercombine=and&filtercount=5&showfilters=1&filtercombine=and&field1=buildname&compare1=61&value1=Trilinos-atdm-waterman-cuda-9.2-release-debug&field2=testname&compare2=65&value2=PanzerAdaptersSTK_&field3=site&compare3=61&value3=waterman&field4=buildstarttime&compare4=84&value4=2018-11-27T00%3A00%3A00&field5=buildstarttime&compare5=83&value5=2018-11-26T00%3A00%3A00) the tests:
* [PanzerAdaptersSTK_MixedCurlLaplacianExample](https://testing.sandia.gov/cdash/testDetails.php?test=60054478&build=4214970)
* [PanzerAdaptersSTK_MixedCurlLaplacianExample-ConvTest-Tri-Order-1](https://testing.sandia.gov/cdash/testDetails.php?test=60054486&build=4214970)
* [PanzerAdaptersSTK_MixedCurlLaplacianExample-ConvTest-Tri-Order-2](https://testing.sandia.gov/cdash/testDetails.php?test=60054484&build=4214970)
* [PanzerAdaptersSTK_MixedCurlLaplacianExample-ConvTest-Quad-Order-2](https://testing.sandia.gov/cdash/testDetails.php?test=61601464&build=4301598)
* [PanzerAdaptersSTK_MixedCurlLaplacianExample-ConvTest-Quad-Order-3](https://testing.sandia.gov/cdash/testDetails.php?test=60054481&build=4214970)
* [PanzerAdaptersSTK_MixedCurlLaplacianMultiblockExample-ConvTest-Quad-Order-1](https://testing.sandia.gov/cdash/testDetails.php?test=61601467&build=4301598)
* [PanzerAdaptersSTK_CurlLaplacianExample-ConvTest-Quad-Order-2](https://testing.sandia.gov/cdash/testDetails.php?test=60054475&build=4214970)
* [PanzerAdaptersSTK_CurlLaplacianExample-ConvTest-Quad-Order-4](https://testing.sandia.gov/cdash/testDetails.php?test=61601465&build=4301598)
* [PanzerAdaptersSTK_CurlLaplacianMultiblockExample-ConvTest-Quad-Order-1](https://testing.sandia.gov/cdash/testDetails.php?test=61601462&build=4301598)
are failing in the build:
* Trilinos-atdm-waterman-cuda-9.2-release-debug
Test names above link to the test output
## Current Status on CDash
The current status of these tests/builds for the current testing day can be found [here](https://testing.sandia.gov/cdash/queryTests.php?project=Trilinos&filtercombine=and&filtercount=3&showfilters=1&filtercombine=and&field1=buildname&compare1=61&value1=Trilinos-atdm-waterman-cuda-9.2-release-debug&field2=testname&compare2=65&value2=PanzerAdaptersSTK_&field3=site&compare3=61&value3=waterman)
## Steps to Reproduce
One should be able to reproduce this failure on waterman as described in:
* https://github.com/trilinos/Trilinos/blob/develop/cmake/std/atdm/README.md
More specifically, the commands given for waterman are provided at:
* https://github.com/trilinos/Trilinos/blob/develop/cmake/std/atdm/README.md#waterman
The exact commands to reproduce this issue should be:
```
$ cd <some_build_dir>/
$ source $TRILINOS_DIR/cmake/std/atdm/load-env.sh Trilinos-atdm-waterman-cuda-9.2-release-debug
$ cmake \
-GNinja \
-DTrilinos_CONFIGURE_OPTIONS_FILE:STRING=cmake/std/atdm/ATDMDevEnv.cmake \
-DTrilinos_ENABLE_TESTS=ON -DTrilinos_ENABLE_Panzer=ON \
$TRILINOS_DIR
$ make NP=16
$ bsub -x -Is -n 20 ctest -j20
```Keep promoted "ATDM" builds of Trilinos cleanhttps://gitlab.osti.gov/jmwille/Trilinos/-/issues/3638Teko_ModALPreconditioner_MPI_1 Failing in build Trilinos-atdm-cee-rhel6-clang...2018-12-20T20:25:33ZJames WillenbringTeko_ModALPreconditioner_MPI_1 Failing in build Trilinos-atdm-cee-rhel6-clang-5.0.1-openmpi-1.10.2-serial-static-opt*Created by: fryeguy52*
CC: @trilinos/teko , @srajama1 (Trilinos Linear Solvers Product Lead), @bartlettroscoe
## Next Action Status
With the merge of PR #4079 to 'develop' on 12/19/2018, this test `Teko_ModALPreconditioner_MPI...*Created by: fryeguy52*
CC: @trilinos/teko , @srajama1 (Trilinos Linear Solvers Product Lead), @bartlettroscoe
## Next Action Status
With the merge of PR #4079 to 'develop' on 12/19/2018, this test `Teko_ModALPreconditioner_MPI_1` is disabled and shown missing in the build `Trilinos-atdm-cee-rhel6-clang-5.0.1-openmpi-1.10.2-serial-static-opt ` on 12/20/2018.
## Description
As shown in [this query](https://testing.sandia.gov/cdash-dev-view/queryTests.php?project=Trilinos&date=2018-10-15&filtercount=2&showfilters=1&filtercombine=and&field1=buildname&compare1=65&value1=Trilinos-atdm-cee-rhel6-&field2=status&compare2=62&value2=passed) the tests:
* Teko_ModALPreconditioner_MPI_1
are failing in the builds:
* Trilinos-atdm-cee-rhel6-clang-opt-serial
failing from a seg fault:
```
[ceerws1113:36105] *** Process received signal ***
[ceerws1113:36105] Signal: Segmentation fault (11)
[ceerws1113:36105] Signal code: Address not mapped (1)
[ceerws1113:36105] Failing at address: (nil)
```
## Steps to Reproduce
One should be able to reproduce this failure any CEE LAN RHEL6 SRN machine as described in:
* https://github.com/trilinos/Trilinos/blob/develop/cmake/std/atdm/README.md
More specifically, the commands given for the CEE LAN RHEL6 SRN machines are provided at:
* https://github.com/trilinos/Trilinos/blob/develop/cmake/std/atdm/README.md#cee-rhel6-environment
The exact commands to reproduce this issue should be:
```
$ cd <some_build_dir>/
$ source $TRILINOS_DIR/cmake/std/atdm/load-env.sh cee-rhel6-clang-opt-serial
$ cmake \
-GNinja \
-DTrilinos_CONFIGURE_OPTIONS_FILE:STRING=cmake/std/atdm/ATDMDevEnv.cmake \
-DTrilinos_ENABLE_TESTS=ON -DTrilinos_ENABLE_Teko=ON \
$TRILINOS_DIR
$ make NP=16
$ ctest -j16
```Initial cleanup of new ATDM builds of Trilinoshttps://gitlab.osti.gov/jmwille/Trilinos/-/issues/2410Test TeuchosNumerics_LAPACK_test_MPI_1 fails in all 'debug' builds on power8 ...2018-12-19T14:42:14ZJames WillenbringTest TeuchosNumerics_LAPACK_test_MPI_1 fails in all 'debug' builds on power8 'ride'*Created by: bartlettroscoe*
**CC:** @trilinos/teuchos
## Next Action Status:
PR #2447 was merged on 3/23/2018 which disabled the test. PR #4064 which enables the whole test `TeuchosNumerics_LAPACK_test_MPI_1` but disables the ...*Created by: bartlettroscoe*
**CC:** @trilinos/teuchos
## Next Action Status:
PR #2447 was merged on 3/23/2018 which disabled the test. PR #4064 which enables the whole test `TeuchosNumerics_LAPACK_test_MPI_1` but disables the single unit test for `STEQR()` merged to 'develop' on 12/18/2018. Next: Watch for test running and passing (minus `STEQR()` unit test) on 'release-debug' and 'opt' builds on 'white', 'ride', and 'waterman' on 12/19/2018 ...
## Description
The test `TeuchosNumerics_LAPACK_test_MPI_1` segfaults on the 'debug' builds `Trilinos-atdm-white-ride-cuda-debug` and `Trilinos-atdm-white-ride-gnu-debug-openmp` on 'ride' and 'white' but passes in all of the 'opt' builds on these same machines as well as for all of the builds on `hansen` as shown this morning in:
* https://testing-vm.sandia.gov/cdash/queryTests.php?project=Trilinos&date=2018-03-17&filtercombine=and&filtercombine=and&filtercount=2&showfilters=1&filtercombine=and&field1=buildname&compare1=65&value1=Trilinos-atdm-&field2=testname&compare2=61&value2=TeuchosNumerics_LAPACK_test_MPI_1
The failing tests all show segfaults showing the output:
```
Teuchos in Trilinos 12.13 (Dev)
GESV test ... passed!
LAPY2 test ... passed!
--------------------------------------------------------------------------
mpiexec noticed that process rank 0 with PID 16320 on node white24 exited on signal 11 (Segmentation fault).
--------------------------------------------------------------------------
```
What is interesting is that this test only failed in all of the Trilinos builds that were done yesterday in the query:
* https://testing-vm.sandia.gov/cdash/queryTests.php?project=Trilinos&date=2018-03-16&filtercombine=and&filtercount=3&showfilters=1&filtercombine=and&field1=buildname&compare1=65&value1=Trilinos-atdm-&field2=testname&compare2=61&value2=TeuchosNumerics_LAPACK_test_MPI_1&field3=status&compare3=62&value3=passed
May this be the same error reported in #1208 that we basically gave up on?
## Steps to Reproduce
Following the instructions at:
* https://github.com/trilinos/Trilinos/blob/develop/cmake/std/atdm/README.md#ridewhite
one can reproduce this failing test by enabling the Teuchos package for the builds `gnu-debug-openmp` or `cuda-debug` and running the failing test.
## Related issues
* Related to: #1208?
Initial cleanup of new ATDM builds of Trilinoshttps://gitlab.osti.gov/jmwille/Trilinos/-/issues/2092New test TeuchosCore_testTeuchosTestForTermination failing in clean build Lin...2017-12-21T18:05:52ZJames WillenbringNew test TeuchosCore_testTeuchosTestForTermination failing in clean build Linux-gcc-4.9.3-SERIAL_Release_gcc_4.9.3__DEV*Created by: bartlettroscoe*
**CC:** @trilinos/framework, @trilinos/teuchos
## Description:
The new test `TeuchosCore_testTeuchosTestForTermination` added yesterday (see #2089) is failing in the clean build `Linux-gcc-4.9.3-SERIA...*Created by: bartlettroscoe*
**CC:** @trilinos/framework, @trilinos/teuchos
## Description:
The new test `TeuchosCore_testTeuchosTestForTermination` added yesterday (see #2089) is failing in the clean build `Linux-gcc-4.9.3-SERIAL_Release_gcc_4.9.3__DEV` shown at:
* https://testing.sandia.gov/cdash/testDetails.php?test=43561818&build=3286245
What is strange is that the test STDOUT is printing:
```
Teuchos::GlobalMPISession::GlobalMPISession(): started serial run
/scratch/trilinos/workspace/trilinos-folder/Trilinos_generic_nightly/SERIAL_Release_gcc_4.9.3__DEV/Trilinos/packages/teuchos/core/test/MemoryManagement/testTeuchosTestForTermination.cpp:63:
Terminate test that evaluated to true: GlobalMPISession::getRank() == terminate_on_procid
Bingo, we are terminating on procid == terminate_on_procid = 0!
terminate called without an active exception
```
but "Fail Reason" says: "Required regular expression found.Regex=[Bingo, we are terminating on procid == terminate_on_procid = 0"
But if you look at that STDOUT, that output is being printed! What the heck?
The only other build so far that is showing this same failure is the Experimental build ` Linux-GCC-4.9.3-RELEASE_DEV_KokkosKernels_Experimental` (which is also a serial build) with the test results shown at:
* https://testing.sandia.gov/cdash/testDetails.php?test=43561320&build=3286162
And that test is exactly the same!
## Related Issues
* Related to: #1303, #2089
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/1947ShyLU_Node and ShyLU_DD configure failures after TriBITS update2017-11-07T17:07:41ZJames WillenbringShyLU_Node and ShyLU_DD configure failures after TriBITS update*Created by: bartlettroscoe*
**CC:** @trilinos/shylu
**Description:**
After my TriBITS update in the commit 9403c2a (which added stronger checking for correct usage of TriBITS, see TriBITSPub/TriBITS#200 and TriBITSPub/TriBITS#23...*Created by: bartlettroscoe*
**CC:** @trilinos/shylu
**Description:**
After my TriBITS update in the commit 9403c2a (which added stronger checking for correct usage of TriBITS, see TriBITSPub/TriBITS#200 and TriBITSPub/TriBITS#232), the ShyLU_Node and ShyLU_DD packages showed configure failures for all of the builds today as shown at:
* https://testing.sandia.gov/cdash/index.php?project=Trilinos&date=2017-11-03&display=project&filtercount=3&showfilters=1&filtercombine=and&field1=subprojects&compare1=93&value1=ShyLU_Node&field2=subprojects&compare2=93&value2=ShyLU_DD&field3=subprojects&compare3=93&value3=ShyLU
These show the configure failures like:
```
Processing enabled package: ShyLU_Node (HTS, Tacho, Tests, Examples)
CMake Error at /scratch/trilinos/workspace/trilinos-folder/Trilinos_generic_nightly/Trilinos/cmake/tribits/core/package_arch/TribitsPackageMacros.cmake:744 (MESSAGE):
Must call TRIBITS_PACKAGE_DECL(), TRIBITS_PROCESS_SUBPACKAGES()and
TRIBITS_PACKAGE_DEF before TRIBITS_PACKAGE_POSTPROCESS(). Because this
package has subpackages you cannot use TRIBITS_PACKAGE() you must call
these in the following order: TRIBITS_PACKAGE_DECL
TRIBITS_PROCESS_SUBPACKAGES TRIBITS_PACKAGE_DEF TRIBITS_PACKAGE_POSTPROCESS
in file:
/scratch/trilinos/workspace/trilinos-folder/Trilinos_generic_nightly/SERIAL_Release_gcc_4.9.3__DEV/Trilinos/packages/shylu/shylu_node/CMakeLists.txt
Call Stack (most recent call first):
packages/shylu/shylu_node/CMakeLists.txt:23 (TRIBITS_PACKAGE_POSTPROCESS)
```
The fix is trivial and I am working to test and push it now.
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/1674Add Wiki page about GitHub notifications2017-09-01T23:22:59ZJames WillenbringAdd Wiki page about GitHub notifications*Created by: maherou*
GitHub notifications are essential for communication, especially for large projects. However, setting up your notifications and dealing with troubleshooting can be challenging. The wiki page:
https://github.co...*Created by: maherou*
GitHub notifications are essential for communication, especially for large projects. However, setting up your notifications and dealing with troubleshooting can be challenging. The wiki page:
https://github.com/trilinos/Trilinos/wiki/GitHub-Notifications
attempts to address the challenges.https://gitlab.osti.gov/jmwille/Trilinos/-/issues/1176Analysis of configure and build failures due to KokkosKernels pushes on March...2017-03-23T17:54:16ZJames WillenbringAnalysis of configure and build failures due to KokkosKernels pushes on March 1-2 reported in #1099*Created by: bartlettroscoe*
**Description:**
This story is to analyze the KokkosKernels commits pushed on March 1-2, 2017 that broke the configure and build of Trilinos that was reported in #1099 and see if usage of the [checkin-tes...*Created by: bartlettroscoe*
**Description:**
This story is to analyze the KokkosKernels commits pushed on March 1-2, 2017 that broke the configure and build of Trilinos that was reported in #1099 and see if usage of the [checkin-test-sems.sh](https://github.com/trilinos/Trilinos/wiki/Policies-%7C-Safe-Checkin-Testing) script could have avoided the failures (and resulting consequences) if it had been used for the test and push (which would have stopped the push).
This started when a set of commits were pushed to the Trilinos 'develop' branch on March 1 with the top commit 97ed757 being:
```
97ed757 [Wed Mar 1 08:24:01 2017 -0700] <crtrott@sandia.gov>
Kokkos-Kernels: Adding Kokkos-Kernels as a stand-alone package
```
as shown by the CI build:
* http://testing.sandia.gov/cdash/viewConfigure.php?buildid=2766164
That version of Trilinos failed to configure as shown on that CI build iteration.
Later that day, issue #1099 was created by an important Trilinos customer and it resulted in 35 comments that involved 9 people in that issue before it was resolved.
An attempt to fix this problem was pushed later that day with the top commit de7ac5a being:
```
de7ac5a [Wed Mar 1 13:17:16 2017 -0700] <mhoemme@sandia.gov>
KokkosKernels: Fix #1099
```
as shown by the CI build:
* http://testing.sandia.gov/cdash/viewConfigure.php?buildid=2766462
That version passed the configure but resulted in many build failures.
The build was not finally fixed until March 2 as shown at:
* http://testing.sandia.gov/cdash/viewConfigure.php?buildid=2767860
Could the usage of the checkin-test-sems.sh script have caught these problems and stop the pushes that broke the configure and build of Trilinos over these two days?
**CC:** @trilinos/framework, @bathmatt, @crtrott, @mhoemmen
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/546Make Trilinos' use of ETI consistent2017-06-02T21:07:25ZJames WillenbringMake Trilinos' use of ETI consistent*Created by: bmpersc*
Trilinos wants to move to ETI to be enabled by default. This requires that all packages use the same ETI system. Currently there are 3, maybe more in Tpetra, Muelu and ifpack2. The goal is to make sure that turning...*Created by: bmpersc*
Trilinos wants to move to ETI to be enabled by default. This requires that all packages use the same ETI system. Currently there are 3, maybe more in Tpetra, Muelu and ifpack2. The goal is to make sure that turning on ETI will not make obscure builds fail due to combining packages that aren't normally during our testing.
Reduce build times for Trilinoshttps://gitlab.osti.gov/jmwille/Trilinos/-/issues/372Revise new developer checklist2016-10-05T18:59:41ZJames WillenbringRevise new developer checklist*Created by: jwillenbring*
**CC:** @trilinos/framework
**Description:**
The new developer checklist needs to be revised. In particular, to be up-to-date with the move to GitHub. Mentors can now decide when new students should get pus...*Created by: jwillenbring*
**CC:** @trilinos/framework
**Description:**
The new developer checklist needs to be revised. In particular, to be up-to-date with the move to GitHub. Mentors can now decide when new students should get push access.
https://software.sandia.gov/trilinos/developer/sqp/checklists/newTrilinosDeveloper201508.txt
**Tasks:**
1. Initial update of checklist [Done]
2. Have set out for review and collect updates ... IN PROGRESS ...
3. Publish checklist to final location ...
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/330Create GPU-capable Thyra/Tpetra adapters for linear iterative algorithms2017-06-30T22:13:34ZJames WillenbringCreate GPU-capable Thyra/Tpetra adapters for linear iterative algorithms*Created by: bartlettroscoe*
**Next Action Status:**
PR #1442 implementing this merged to 'develop'. Next: Wait for feedback from automated testing and customers ...
**CC List:** @rppawlo
**Description:**
The current Thyr...*Created by: bartlettroscoe*
**Next Action Status:**
PR #1442 implementing this merged to 'develop'. Next: Wait for feedback from automated testing and customers ...
**CC List:** @rppawlo
**Description:**
The current Thyra-Tpetra adaptors simply inherit from Thyra::SpmdMultiVectorBase, implementing the getNonconstLocalDataImpl() and getLocalDataImpl() methods to get data pointers to the Tpetra data. For a GPU node, this has the effect of copying the data to the host, so that Thyra (or more likely RTOp) can manipulate it there, and then copy it back (if it is modified). What we need for this work is a new Thyra-Tpetra adapter that will call native Tpetra methods on Tpetra::MultiVector.
The initial scope of this work will be to just avoid calling RTOps in the linear iterative algorithms that are in Belos and Anasazi. In particular, the goal is to avoid copies of data and RTOps for:
- Anasazi::BlockKrylovSchur and Anasazi::BlockKrylovSchurSolverManager
- Belos::BlockGmresSolMgr via Stratimikos DefaultLinearSolverBuilder
A tentative list of arithmetic methods needed by Anasazi BKS is:
- MVT::SetBlock(), which calls Thyra::assign()
- MVT::MvRandom(), which calls Thyra::randomize()
- MVT::MvTimesMatAddMv(), which calls MultiVectorBase::apply()
- MVT::MvNorm(), which calls Thyra::norms_2()
- MVT::MvAddMv(), which calls Thyra::linear_combination()
Belos::BlockGmresIter and BlockGmresSolMgr require similar functionality.
It appears that the adaptor between Thyra::LinearOpBase and Tpetra::Operator is fine.
NOTE: This is the GitHub version of the Trilinos Bugzilla ticket [#5837](https://software.sandia.gov/bugzilla/show_bug.cgi?id=5837) "Create GPU-capable Thyra adaptors for Tpetra". This work will now be tracked in this GitHub issue and only refer back to the Bugzilla ticket for historical purposes.
**Definition of Done:**
???
**Tasks:**
1. Provide complete accounting of all of the Belos MultiVector traits functions that are called for the targeted Belos GMRES solver (set up a test case for this purpose). (Hint: Run profiler or use Teuchos Time Monitor to determine this.)
2. Determine plan for adding minimal new virtual functions to Thyra::MultiVectorBase to replace the majority of the calls to RTOps.
3. Add new pure virtual functions to the Thyra::MultiVectorBase interface assign() and linearCombination() to replace as many RTOps as possible.
4. ???
NOX/Belos/Thyra/Tpetra GPUhttps://gitlab.osti.gov/jmwille/Trilinos/-/issues/229Make Teuchos Memory Management Classes thread-safe2017-08-07T02:09:35ZJames WillenbringMake Teuchos Memory Management Classes thread-safe*Created by: bartlettroscoe*
This story is to address the long-standing problem that the Teuchos Memory management Classes which use reference-counting are not thread safe.
CC: @MicheldeMessieres, @jwillenbring,
**Next Action Status:...*Created by: bartlettroscoe*
This story is to address the long-standing problem that the Teuchos Memory management Classes which use reference-counting are not thread safe.
CC: @MicheldeMessieres, @jwillenbring,
**Next Action Status:** See tasks ...
**Tasks:**
1. Initial development and testing for multi-thread correctness [Done]
2. **Add configure time switch for thread safety:** Define configure-time options `Trilinos_ENABLE_THREAD_SAFE` and `Teuchos_ENABLE_THREAD_SAFE` (latter is given the given the default of the former value).
3. **Turn off for Trilinos_ENABLE_CXX11=OFF**: That is, set `Teuchos_ENABLE_THREAD_SAFE=OFF` in this case. Run full Trilinos test suite with `-DTrilinos_ENABLE_CXX11=OFF`.
4. **Update the Teuchos test suite:**
- **Inform CTest of number of threads for thread-safe tests:** Figure this out at configure time and then set `NUM_TOTAL_CORES_USED` (see [TRIBITS_ADD_TEST())(https://tribits.org/doc/TribitsDevelopersGuide.html#formal-arguments-tribits-add-test))
- **Make pre-push `BASIC` test suite fast:** Make the longer running threading tests `NIGHTLY`.
5. **Performance testing:**
- For builds:
- `-DCMAKE_BUILD_TYPE=RELEASE -DTrilinos_ENABLE_DEBUG=ON` (`Trilinos_ENABLE_THREAD_SAFE` on and off)
- `-DCMAKE_BUILD_TYPE=RELEASE -DTrilinos_ENABLE_DEBUG=OFF` (`Trilinos_ENABLE_THREAD_SAFE` on and off)
- For compilers:
- GCC version 4.8.x .
- Intel version 15.x
- Clang X
- Run Trilinos (nearly full) test suite with and without thread-safety turned on.
- Run Nalu, Albany, and Drekar test suites with thread safety on and off and see the performance impact with debug-mode checking turned on.
- Request report from Cedric about usage and performance.
- If performance okay, continue. Otherwise, decide what to do.
6. **Disallow throwing exceptions from destructors:** We just need to disallow exceptions and make Teuchos MM classes abort in destructors when errors occur. Update unit tests for the case of circular references and exceptions. Need to provide `TEUCHOS_ABORT_IF(<condition>)` that will print and then call abort.
7. **Merge into develop branch with Trilinos_ENABLE_THREAD_SAFE=OFF by default**:
- Update teuchos/ReleaseNotes.txt to discuss exception destructor difference.
- Announce time schedule for turning this on by default.
8. **Update documentation / Code review:**
- Update unit test documentation: With final tests in place, will create a uniformly formatted summary for each in code to describe it’s purpose.
- Update RCP documentation: Need to update RCP documents to reflect these changes
- Ross reviews code, tests, and updated documentation.
9. **Turn on Trilinos_ENABLE_THREAD_SAFE=OFF by default:**
- Update teuchos/ReleaseNotes.txt
- Send out announcement
10. **Other considerations and improvements:** (move to new stories?)
1. **Review Array.h mutex implementation:** This was new code I added after our last review to make Array.h thread safe - I have implemented suggested tests we discussed on Github.
2. **Discuss plan for debug detection of dangling weak ptr.** Debug builds have checks to validate weak ptrs but those checks can fail if another thread kills the data. I’ve got tests in place which detect and demonstrate this issue but need to discuss further how we would like to address this.
3. **Consider additional changes for ArrayView, ArrayRCP, Tuple, Ptr**: Implemented fairly limited sanity checks on these.
4. **Weak to strong conversion:** Have code in place which implements thread safe upgrade of a weak ptr to a strong ptr, along with a unit test, but the role of this is unclear at the moment.
5. **Make tests have inverted case**: Tests should demonstrate they can detect thread problems when the fix is not applied - the inverted case. I’ve got some #defines set up to do this but wanted to discuss how to best organize those. Many of the inverted tests will need separate main functions.