Trilinos issueshttps://gitlab.osti.gov/jmwille/Trilinos/-/issues2019-05-01T19:20:49Zhttps://gitlab.osti.gov/jmwille/Trilinos/-/issues/4847KokkosKernels_graph tests timing out in ATDM builds2019-05-01T19:20:49ZJames WillenbringKokkosKernels_graph tests timing out in ATDM builds*Created by: fryeguy52*
CC: @trilinos/kokkoskernels, @kddevin (Trilinos Data Services Product Lead), @bartlettroscoe, @fryeguy52
<Checklist>
<???: Add label "ATDM">
<???: Add label "bug"?>
<???: Add label for affected packages (e...*Created by: fryeguy52*
CC: @trilinos/kokkoskernels, @kddevin (Trilinos Data Services Product Lead), @bartlettroscoe, @fryeguy52
<Checklist>
<???: Add label "ATDM">
<???: Add label "bug"?>
<???: Add label for affected packages (e.g. "MueLu", "Tpetra", "Kokkos", etc.)>
<???: Add milestone "Initial cleanup of new ATDM builds of Trilinos" or "Keep promoted ATDM builds of Trilinos clean">
<???: Once GitHub Issue is created, add entries for tests to TrilinosATDMStatus/*.csv files>
<???: Add label "PA: ???Project Area???" (e.g. "PA: Linear Solvers", "PA: Data Services")>
## Next Action Status
<status-and-or-first-action>
## Description
As shown in [this query](https://testing.sandia.gov/cdash/queryTests.php?project=Trilinos&filtercombine=and&filtercombine=&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercount=6&showfilters=1&filtercombine=and&field1=buildname&compare1=65&value1=Trilinos-atdm-&field2=testname&compare2=65&value2=KokkosKernels_graph_&field3=testname&compare3=66&value3=_MPI_1&field4=status&compare4=62&value4=Passed&field5=buildstarttime&compare5=83&value5=2019-04-07&field6=buildstarttime&compare6=84&value6=2019-04-10) the tests:
* KokkosKernels_graph_openmp_MPI_1
* KokkosKernels_graph_serial_MPI_1
started timing out on 2019-04-07 in the builds:
* Trilinos-atdm-hansen-shiller-gnu-debug-serial
* Trilinos-atdm-waterman-cuda-9.2-debug
* Trilinos-atdm-white-ride-cuda-9.2-gnu-7.2.0-debug
* Trilinos-atdm-white-ride-gnu-7.2.0-openmp-debug
## Current Status on CDash
Failures for the current testing day can be found [here](https://testing.sandia.gov/cdash/queryTests.php?project=Trilinos&filtercombine=and&filtercombine=&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercount=6&showfilters=1&filtercombine=and&field1=buildname&compare1=65&value1=Trilinos-atdm-&field2=testname&compare2=65&value2=KokkosKernels_graph_&field3=testname&compare3=66&value3=_MPI_1&field4=status&compare4=62&value4=Passed&field5=buildstarttime&compare5=83&value5=yesterday&field6=buildstarttime&compare6=84&value6=today)
## Steps to Reproduce
One should be able to reproduce this failure on ride or white as described in:
* https://github.com/trilinos/Trilinos/blob/develop/cmake/std/atdm/README.md
More specifically, the commands given for ride or white are provided at:
* https://github.com/trilinos/Trilinos/blob/develop/cmake/std/atdm/README.md#ridewhite
The exact commands to reproduce this issue should be:
```
$ cd <some_build_dir>/
$ source $TRILINOS_DIR/cmake/std/atdm/load-env.sh Trilinos-atdm-white-ride-gnu-7.2.0-openmp-debug
$ cmake \
-GNinja \
-DTrilinos_CONFIGURE_OPTIONS_FILE:STRING=cmake/std/atdm/ATDMDevEnv.cmake \
-DTrilinos_ENABLE_TESTS=ON -DTrilinos_ENABLE_KokkosKernels=ON \
$TRILINOS_DIR
$ make NP=16
$ bsub -x -Is -q rhel7F -n 16 ctest -j16
```
Keep promoted "ATDM" builds of Trilinos cleanhttps://gitlab.osti.gov/jmwille/Trilinos/-/issues/4852Tpetra: Make CrsGraph implement the "new" DistObject interface 2019-04-12T19:09:48ZJames WillenbringTpetra: Make CrsGraph implement the "new" DistObject interface *Created by: mhoemmen*
@trilinos/tpetra
## Current Behavior
`Tpetra::CrsGraph` currently implements the "old" DistObject interface (copyAndPermute, packAndPrepare, unpackAndCombine) instead of the "new" DistObject interface (copy...*Created by: mhoemmen*
@trilinos/tpetra
## Current Behavior
`Tpetra::CrsGraph` currently implements the "old" DistObject interface (copyAndPermute, packAndPrepare, unpackAndCombine) instead of the "new" DistObject interface (copyAndPermuteNew, packAndPrepareNew, unpackAndCombineNew).
## Motivation and Context
This blocks deprecation and removal of the "old" DistObject interface.
## Definition of Done
- [ ] Implement copyAndPermuteNew, packAndPrepareNew, and unpackAndCombineNew
- [ ] Implement useNewInterface and have it return true
- [ ] Make sure all the tests pass
## Possible Solution
@tjfulle has spent quite a bit of effort on thread-parallel CrsGraph pack and unpack. It looks like he did most of the underlying work already. This is more about getting the interfaces to match and plugging everything in.
## Related Issues
* Blocks https://github.com/trilinos/Trilinos/issues/4853 https://gitlab.osti.gov/jmwille/Trilinos/-/issues/4850MueLu: update LTG Matrix kernels in TpetraExt2019-04-10T20:53:18ZJames WillenbringMueLu: update LTG Matrix kernels in TpetraExt*Created by: jjellio*
Update the TpetraExt (LTG) kernels to use improved copies and memory management.
These changes were not propagated from the work done last fall. This issue mirrors the PR being submitted.
@trilinos/muelu
@c...*Created by: jjellio*
Update the TpetraExt (LTG) kernels to use improved copies and memory management.
These changes were not propagated from the work done last fall. This issue mirrors the PR being submitted.
@trilinos/muelu
@csiefer2
## Expectations
- [x] The kernels will use bulk threaded copies for copy-out
- [x] The kernels will compute the rowptr in-place (reduced memory overhead)
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/4843Tempus: Separate Stepper Header and Source2019-04-11T21:48:09ZJames WillenbringTempus: Separate Stepper Header and Source*Created by: ccober6*
Cleanup the Stepper base class and separate the header from the source code.
@trilinos/tempus
## Expectations
Nothing should change. Just code shuffling to improve code quality.*Created by: ccober6*
Cleanup the Stepper base class and separate the header from the source code.
@trilinos/tempus
## Expectations
Nothing should change. Just code shuffling to improve code quality.https://gitlab.osti.gov/jmwille/Trilinos/-/issues/4836Tempus: Improve Interval Output Setup2019-04-11T21:48:56ZJames WillenbringTempus: Improve Interval Output Setup*Created by: ccober6*
In the output interval and the screen output interval, all the intervals are placed in a vector along with the specified output times or indices, and then sorted. This allowed all the output times and indices to b...*Created by: ccober6*
In the output interval and the screen output interval, all the intervals are placed in a vector along with the specified output times or indices, and then sorted. This allowed all the output times and indices to be in one vector and searched. However when the number of time steps reaches millions, the sort is far too expensive. This will rework the interval specification so that it is not part of the vector or sorting.
@trilinos/tempus
## Expectations
Work the same as previously but without the costly sorting.https://gitlab.osti.gov/jmwille/Trilinos/-/issues/4840Tpetra: Deprecate MultiVectorFiller2019-04-11T18:23:38ZJames WillenbringTpetra: Deprecate MultiVectorFiller*Created by: csiefer2*
Because Tpetra::FEMultiVector is so much better...*Created by: csiefer2*
Because Tpetra::FEMultiVector is so much better...https://gitlab.osti.gov/jmwille/Trilinos/-/issues/4829KokkosKernels broke Albany build on Ride2019-04-10T14:34:42ZJames WillenbringKokkosKernels broke Albany build on Ride*Created by: ikalash*
It appears some changes to KokkosKernels broke Albany's Ride build over the weekend. Here is the error:
```
58%] Building C object packages/aztecoo/src/CMakeFiles/aztecoo.dir/az_gsumd_puma.c.o ...*Created by: ikalash*
It appears some changes to KokkosKernels broke Albany's Ride build over the weekend. Here is the error:
```
58%] Building C object packages/aztecoo/src/CMakeFiles/aztecoo.dir/az_gsumd_puma.c.o /.../repos/Trilinos/packages/kokkos-kernels/src/sparse/impl/KokkosSparse_spgemm_impl_color.hpp(516): error: namespace "KokkosGraph::Experimental" has no member "d2_graph_color"
--
```
http://cdash.sandia.gov/CDash-2-3-0/viewBuildError.php?buildid=83384
Could someone please have a look?
@trilinos/kokkos-kernels https://gitlab.osti.gov/jmwille/Trilinos/-/issues/4827Tpetra::Details::copyConvert doesn't build for conversions between Kokkos::co...2019-04-16T14:01:50ZJames WillenbringTpetra::Details::copyConvert doesn't build for conversions between Kokkos::complex<float> and double*Created by: mhoemmen*
@trilinos/tpetra
This breaks `Tpetra::deep_copy` between `MultiVector<std::complex<float>, ...>` and `MultiVector<double, ...>`.*Created by: mhoemmen*
@trilinos/tpetra
This breaks `Tpetra::deep_copy` between `MultiVector<std::complex<float>, ...>` and `MultiVector<double, ...>`.https://gitlab.osti.gov/jmwille/Trilinos/-/issues/4821Tempus: Add method to set Tableaus.2019-04-11T21:48:37ZJames WillenbringTempus: Add method to set Tableaus.*Created by: ccober6*
@trilinos/tempus
## Motivation and Context
To better setup RK methods, it is nice to set the Butcher Tableau directly instead of through a ParameterList.
## Solution
Add set method for the tableaus.*Created by: ccober6*
@trilinos/tempus
## Motivation and Context
To better setup RK methods, it is nice to set the Butcher Tableau directly instead of through a ParameterList.
## Solution
Add set method for the tableaus.https://gitlab.osti.gov/jmwille/Trilinos/-/issues/4809Framework: Non-framework machines no longer able to report to Cdash2019-04-09T23:16:48ZJames WillenbringFramework: Non-framework machines no longer able to report to Cdash*Created by: csiefer2*
And for that matter, the entire Nightly track is gone. SAD!
(Originally noticed by @lucbv)
@trilinos/framework
@trilinos/tpetra This will impact deprecation work after the release if it is not resolved...*Created by: csiefer2*
And for that matter, the entire Nightly track is gone. SAD!
(Originally noticed by @lucbv)
@trilinos/framework
@trilinos/tpetra This will impact deprecation work after the release if it is not resolved by then.https://gitlab.osti.gov/jmwille/Trilinos/-/issues/4813Zoltan2: example passes extra arguments to Kokkos::View constructor2019-04-06T09:36:26ZJames WillenbringZoltan2: example passes extra arguments to Kokkos::View constructor*Created by: mhoemmen*
@trilinos/zoltan2 @kddevin
`Trilinos/packages/zoltan2/example/block/kokkosBlock.cpp`, line 147, creates a 1-D `Kokkos::View`, but passes it two run-time dimensions. The second dimension is zero, which is why ...*Created by: mhoemmen*
@trilinos/zoltan2 @kddevin
`Trilinos/packages/zoltan2/example/block/kokkosBlock.cpp`, line 147, creates a 1-D `Kokkos::View`, but passes it two run-time dimensions. The second dimension is zero, which is why it still builds, but it throws if you disable deprecated Kokkos code.
I am working on a fix.https://gitlab.osti.gov/jmwille/Trilinos/-/issues/4812Setting kokkos-ndevices in the ctest. 2019-04-04T22:32:41ZJames WillenbringSetting kokkos-ndevices in the ctest. *Created by: kyungjoo-kim*
A question or a suggestion.
When we run ctest, we in general use 4 mpi ranks. In cuda test (assuming we use ride or white), it would be better if we use ``--kokkos-ndevices=4`` as it can reduce the testing ...*Created by: kyungjoo-kim*
A question or a suggestion.
When we run ctest, we in general use 4 mpi ranks. In cuda test (assuming we use ride or white), it would be better if we use ``--kokkos-ndevices=4`` as it can reduce the testing time on cuda. When I just run the ctest, it does not append the kokkos option. Is there anyway to append the option in nightly tests ? This can also reduce time over failures in PR testing.
@bartlettroscoe @trilinos/framework @crtrott
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/4814Tpetra: Hide all currently deprecated code with TPETRA_ENABLE_DEPRECATED_CODE...2019-04-06T09:36:26ZJames WillenbringTpetra: Hide all currently deprecated code with TPETRA_ENABLE_DEPRECATED_CODE macro*Created by: mhoemmen*
@trilinos/tpetra
- [x] For anything in Tpetra marked with `TPETRA_DEPRECATED`, wrap it in `#ifdef TPETRA_ENABLE_DEPRECATED_CODE ... #endif`.
- [x] Make sure that all downstream code builds with `TPETRA_EN...*Created by: mhoemmen*
@trilinos/tpetra
- [x] For anything in Tpetra marked with `TPETRA_DEPRECATED`, wrap it in `#ifdef TPETRA_ENABLE_DEPRECATED_CODE ... #endif`.
- [x] Make sure that all downstream code builds with `TPETRA_ENABLE_DEPRECATED_CODE` not defined (it is currently defined by default; set the CMake option `Tpetra_ENABLE_DEPRECATED_CODE` (case significant) to `OFF`, in order to build with the macro not defined).
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/4768basic examples not working in ML package2019-04-29T16:48:39ZJames Willenbringbasic examples not working in ML package*Created by: chandu1263*
I built ML package using ml_guide.
And basic examples are not working. Please let me know if the build id wrong or anything else is the issue. I am attaching screenshots.
![Screenshot from 2019-03-30 16-55-36]...*Created by: chandu1263*
I built ML package using ml_guide.
And basic examples are not working. Please let me know if the build id wrong or anything else is the issue. I am attaching screenshots.
![Screenshot from 2019-03-30 16-55-36](https://user-images.githubusercontent.com/33172514/55275473-98378880-530c-11e9-9948-72636c55f59d.png)
![Screenshot from 2019-03-30 16-54-49](https://user-images.githubusercontent.com/33172514/55275474-98d01f00-530c-11e9-84e6-9a15b2c40646.png)
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/4801Many PanzerAdaptersSTK (random?) failures in ATDM sems-rhel7+cuda+complex builds2019-04-20T20:12:51ZJames WillenbringMany PanzerAdaptersSTK (random?) failures in ATDM sems-rhel7+cuda+complex builds*Created by: fryeguy52*
CC: @trilinos/panzer, @mperego (Trilinos Discretizations Product Lead), @bartlettroscoe, @fryeguy52
<Checklist>
<???: Add label "ATDM">
<???: Add label "bug"?>
<???: Add label for affected packages (e.g. "...*Created by: fryeguy52*
CC: @trilinos/panzer, @mperego (Trilinos Discretizations Product Lead), @bartlettroscoe, @fryeguy52
<Checklist>
<???: Add label "ATDM">
<???: Add label "bug"?>
<???: Add label for affected packages (e.g. "MueLu", "Tpetra", "Kokkos", etc.)>
<???: Add milestone "Initial cleanup of new ATDM builds of Trilinos" or "Keep promoted ATDM builds of Trilinos clean">
<???: Once GitHub Issue is created, add entries for tests to TrilinosATDMStatus/*.csv files>
<???: Add label "PA: ???Project Area???" (e.g. "PA: Linear Solvers", "PA: Data Services")>
## Next Action Status
## Description
As shown in [this query](https://testing.sandia.gov/cdash/queryTests.php?project=Trilinos&filtercombine=and&filtercombine=&filtercombine=and&filtercombine=and&filtercombine=and&filtercount=6&showfilters=1&filtercombine=and&field1=buildname&compare1=65&value1=Trilinos-atdm-sems-rhel7-cuda-9.2-Volta70-&field2=testname&compare2=65&value2=PanzerAdaptersSTK_&field3=site&compare3=61&value3=sems-rhel7&field4=buildstarttime&compare4=84&value4=2019-04-03T00%3A00%3A00&field5=buildstarttime&compare5=83&value5=2019-03-21T00%3A00%3A00&field6=status&compare6=61&value6=Failed) these tests:
* PanzerAdaptersSTK_CurlLaplacianExample
* PanzerAdaptersSTK_CurlLaplacianExample-ConvTest-Quad-Order-1
* PanzerAdaptersSTK_CurlLaplacianExample-ConvTest-Quad-Order-2
* PanzerAdaptersSTK_CurlLaplacianExample-ConvTest-Quad-Order-3
* PanzerAdaptersSTK_CurlLaplacianExample-ConvTest-Quad-Order-4
* PanzerAdaptersSTK_CurlLaplacianMultiblockExample-ConvTest-Quad-Order-1
* PanzerAdaptersSTK_MixedCurlLaplacianExample-ConvTest-Quad-Order-1
* PanzerAdaptersSTK_MixedCurlLaplacianExample-ConvTest-Quad-Order-2
* PanzerAdaptersSTK_MixedCurlLaplacianExample-ConvTest-Quad-Order-3
* PanzerAdaptersSTK_MixedCurlLaplacianExample-ConvTest-Tri-Order-1
* PanzerAdaptersSTK_MixedCurlLaplacianExample-ConvTest-Tri-Order-2
* PanzerAdaptersSTK_MixedCurlLaplacianMultiblockExample-ConvTest-Quad-Order-1
* PanzerAdaptersSTK_MixedPoissonExample-ConvTest-Hex-Order-1
* PanzerAdaptersSTK_MixedPoissonExample-ConvTest-Hex-Order-2
* PanzerAdaptersSTK_MixedPoissonExample-ConvTest-Hex-Order-3
* PanzerAdaptersSTK_MixedPoissonMultiblockExample-ConvTest-Hex-Order-1
* PanzerAdaptersSTK_PoissonInterfaceExample_2d_diffsideids_MPI_1
have failed at least once on one of these builds since 2019-03-21:
* Trilinos-atdm-sems-rhel7-cuda-9.2-Volta70-complex-static-release-debug
* Trilinos-atdm-sems-rhel7-cuda-9.2-Volta70-complex-shared-release-debug
## Current Status on CDash
PanzerAdaptersSTK tests that failed on those builds for the current testing day can be found [here](https://testing.sandia.gov/cdash/queryTests.php?project=Trilinos&filtercombine=and&filtercombine=&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercount=6&showfilters=1&filtercombine=and&field1=buildname&compare1=65&value1=Trilinos-atdm-sems-rhel7-cuda-9.2-Volta70-&field2=testname&compare2=65&value2=PanzerAdaptersSTK_&field3=site&compare3=61&value3=sems-rhel7&field4=buildstarttime&compare4=84&value4=today&field5=buildstarttime&compare5=83&value5=yesterday&field6=status&compare6=61&value6=Failed)
## Steps to Reproduce
One should be able to reproduce this failure on with a sems rhel6 environment as described in:
* https://github.com/trilinos/Trilinos/blob/develop/cmake/std/atdm/README.md
More specifically, the commands given for with a sems rhel6 environment are provided at:
* https://github.com/trilinos/Trilinos/blob/develop/cmake/std/atdm/README.md#sems-rhel6-environment
The exact commands to reproduce this issue should be:
```
$ cd <some_build_dir>/
$ source $TRILINOS_DIR/cmake/std/atdm/load-env.sh Trilinos-atdm-sems-rhel7-cuda-9.2-Volta70-complex-static-release-debug
$ cmake \
-GNinja \
-DTrilinos_CONFIGURE_OPTIONS_FILE:STRING=cmake/std/atdm/ATDMDevEnv.cmake \
-DTrilinos_ENABLE_TESTS=ON -DTrilinos_ENABLE_Panzer=ON \
$TRILINOS_DIR
$ make NP=16
$ ctest -j8
```
Initial cleanup of new ATDM builds of Trilinoshttps://gitlab.osti.gov/jmwille/Trilinos/-/issues/4796TriBits: undefined variable breaks nightly builds2019-04-18T17:27:20ZJames WillenbringTriBits: undefined variable breaks nightly builds*Created by: lucbv*
@bartlettroscoe (there is not good team for that type of issues...)
## Expectations
triBits changes should not break current nightly builds.
## Current Behavior
The nightly builds from MueLu are all failing a...*Created by: lucbv*
@bartlettroscoe (there is not good team for that type of issues...)
## Expectations
triBits changes should not break current nightly builds.
## Current Behavior
The nightly builds from MueLu are all failing at configure time due to an undefined variable: `${${PROJECT_NAME}_TRIBITS_DIR}` in file `cmake/tribits/core/utils/MessageWrapper.cmake` at line 45.
## Motivation and Context
This has taken down all the nightly MueLu builds which means that we cannot detect bugs in our specialize and experimental tracks that usually are not tested by ATDM or Continuous builds.
## Possible Solution
Is seems that changes done last week in triBits are to blame, see commit `2283e955`
## Steps to Reproduce
Attempting to run any build using the `cmake/ctest/drivers/{enigma,geminga,rocketman,trappist}` will fail.https://gitlab.osti.gov/jmwille/Trilinos/-/issues/4797Tpetra::MultiVector: Deprecate getDualView & remove its use from Trilinos2019-04-05T20:34:29ZJames WillenbringTpetra::MultiVector: Deprecate getDualView & remove its use from Trilinos*Created by: mhoemmen*
@trilinos/tpetra
- [x] Deprecate `Tpetra::MultiVector::getDualView`.
- [x] Remove its use from Trilinos.
- [x] Explain (in the method's Doxygen documentation) how to get the same functionality.
## Motivat...*Created by: mhoemmen*
@trilinos/tpetra
- [x] Deprecate `Tpetra::MultiVector::getDualView`.
- [x] Remove its use from Trilinos.
- [x] Explain (in the method's Doxygen documentation) how to get the same functionality.
## Motivation and Context
The `getDualView` method exposes an implementation detail of `Tpetra::MultiVector`. This hinders fixing #364 and #333.https://gitlab.osti.gov/jmwille/Trilinos/-/issues/4793Teuchos: Modified parameter lists2019-05-07T13:31:38ZJames WillenbringTeuchos: Modified parameter lists*Created by: danielsjensen1*
<!---
Provide a general summary of the issue in the Title above. If this issue
pertains to a particular package in Trilinos, it's worthwhile to start the
title with "PackageName: ".
-->
<!---
Note ...*Created by: danielsjensen1*
<!---
Provide a general summary of the issue in the Title above. If this issue
pertains to a particular package in Trilinos, it's worthwhile to start the
title with "PackageName: ".
-->
<!---
Note that anything between these delimiters is a comment that will not appear
in the issue description once created. Click on the Preview tab to see what
everything will look like when you submit.
-->
<!---
Feel free to delete anything from this template that is not applicable to the
issue you are submitting.
-->
<!---
Replace <teamName> below with the appropriate Trilinos package/team name.
-->
@trilinos/Teuchos
<!---
Assignees: If you know anyone who should likely tackle this issue, select them
from the Assignees drop-down on the right.
-->
<!---
Labels: Choose any applicable package names from the Labels drop-down on the
right. Additionally, choose a label to indicate the type of issue, for
instance, bug, build, documentation, enhancement, etc.
-->
enhancement
## Expectations
<!---
Tell us what you think should happen, how you think things should work, what
you would like to see in the documentation, etc.
-->
Parameter lists currently don't handle validation of dependent parameters, optional sublists/parameters, and template-like expansion of sublists/parameters.
## Current Behavior
<!---
Tell us how the current behavior fails to meet your expectations in some way.
-->
### Examples of limitations:
#### Validation of dependent parameters
In the time-stepping block below there are three interdependent parameters. The current validation method (validateParametersAndSetDefaults) can ensure that all of the values are valid floats and even positive values but there is not an obvious way to ensure that they are consistent without doing a postprocessing reconciliation step. We would like to attach a reconciliation method to parameter lists to handle such cases.
```
Time Stepping:
Final Time: 1.e-0
Time Step: 1.e-3
Number of Timesteps: ?
```
#### Optional sublists/parameters
In the following Equations block there could be three types of equations specified with different optional parameters. In order to validate these sublists a valid parameter list (let's call it valid_pl) with the correct structure and valid defaults needs to be created and have only the sublists that the user specifies. This is not possible to do without first looking at the user's input parameter list to see which of the optional sublists are included before validation. We propose the attachment of a modifier method to parameter lists that can do this modification to valid_pl before the validation method is run.
```
Equations:
Laguerre:
Order: 1
Hermite:
Order: 2
Hypergeometric:
Order: 0
```
#### Expanding sublists
Often we would like to expand a sublist based on a template and base name as in the example below with Laguerre 1 and Laguerre 2 having the same structure. This also requires parsing the user's input parameter list and creating a new valid parameter list based on the sublists discovered. The modifier method described above can handle this situation in a similar fashion to the 'Optional sublists/parameters' issue described above.
```
Equations:
Laguerre 1:
Order: 1
Laguerre 2:
Order: 2
```
## Motivation and Context
<!---
How has this expectation failure affected you? What are you trying to
accomplish? Why do we need to address this? What does it have to do with
anything? Providing context helps us come up with a solution that is most
useful in the real world.
-->
In scientific programs of significant complexity, the current parameter list validation implementation is not sufficient and significant amounts of code are added to achieve the desired functionality. We propose that these modification and reconciliation methods be standardized and added to the current parameter list functionality.
The pull request https://github.com/trilinos/Trilinos/pull/4834 allows one to attach a modifier to any given parameter (sub)list similar to the way that validators are attached to parameters. (The name "modifier" could be changed maybe even to "validator" if desired.) Each modifier can have a modify and reconcile method. The modify method looks at the user's input parameter list (input_pl) and modifies the validation parameter list (valid_pl) to contain the same structure by handling optional parameters/sublists. After calling the modify method (usually recursively) one would typically call the validateParametersAndSetDefaults method. After all of the parameters exist and are of the correct type the reconcile method can be called (usually recursively) to check that all interdependent parameters are valid. A simple example of this process is shown below. The new tests mentioned in "Possible Solution" contain additional examples.
```
using Teuchos::ParameterListModifier;
using Teuchos::ParameterList;
class TimeSteppingModifier: public ParameterListModifier{
public:
TimeSteppingModifier(): ParameterListModifier("Time Stepping Modifier"){}
void reconcile(ParameterList &pl) const{
double &tf = pl.get<double>("Final Time");
double &dt = pl.get<double>("Time Step");
int &num_dt = pl.get<int>("Number of Time Steps");
double tf_new, dt_new;
int num_dt_new;
if (tf > 0.0 and dt > 0.0){
num_dt_new = std::ceil(tf / dt);
if (num_dt > 0){
TEUCHOS_ASSERT_EQUALITY(num_dt, num_dt_new);
} else{
num_dt = num_dt_new;
}
} else if (dt > 0.0 and num_dt > 0){
tf_new = num_dt * dt;
if (tf > 0.0){
TEUCHOS_ASSERT_EQUALITY(tf, tf_new);
} else{
tf = tf_new;
}
} else if (tf > 0.0 and num_dt > 0){
dt_new = tf / static_cast<double>(num_dt);
if (dt > 0.0){
TEUCHOS_ASSERT_EQUALITY(dt, dt_new);
} else{
dt = dt_new;
}
} else{
throw std::logic_error("Less than two positive time parameters, "
"(""Final Time"", ""Time Step"", ""Number of Timesteps""), given.");
}
}
};
class EquationsModifier: public ParameterListModifier{
public:
EquationsModifier(): ParameterListModifier("Equations Modifier"){}
void modify(ParameterList &pl, ParameterList &valid_pl) const{
int num_expanded = 0;
num_expanded += expandSublistsUsingBaseName("Laguerre", pl, valid_pl);
num_expanded += expandSublistsUsingBaseName("Hermite", pl, valid_pl);
num_expanded += expandSublistsUsingBaseName("Hypergeometric", pl, valid_pl);
if (num_expanded < 1){
throw std::logic_error("There must be at least one equation.");
}
}
};
RCP<TimeSteppingModifier> time_stepping_modifier = rcp(new TimeSteppingModifier());
RCP<EquationsModifier> equations_modifier = rcp(new EquationsModifier());
ParameterList valid_pl = ParameterList("valid_pl");
valid_pl.sublist("Time Stepping", time_stepping_modifier);
valid_pl.sublist("Time Stepping").set("Final Time", 0.0);
valid_pl.sublist("Time Stepping").set("Time Step", 0.0);
valid_pl.sublist("Time Stepping").set("Number of Time Steps", 0);
valid_pl.sublist("Equations", equations_modifier);
valid_pl.sublist("Equations").sublist("Laguerre").set("Order", 0);
valid_pl.sublist("Equations").sublist("Hermite").set("Order", 0);
valid_pl.sublist("Equations").sublist("Hypergeometric").set("Order", 0);
ParameterList pl = ParameterList("pl");
pl.sublist("Time Stepping").set("Final Time", 1.0);
pl.sublist("Time Stepping").set("Time Step", 1e-3);
// If "Number of Time Steps" is set below then it must be consistent with
// "Final Time" and "Time Step" above.
// pl.sublist("Time Stepping").set("Number of Time Steps", 10);
pl.sublist("Equations").sublist("Laguerre 1").set("Order", 3);
pl.sublist("Equations").sublist("Laguerre 2").set("Order", 5);
pl.sublist("Equations").sublist("Hermite 1").set("Order", 0);
pl.modifyParameterList(valid_pl);
pl.validateParametersAndSetDefaults(valid_pl);
pl.reconcileParameterList(valid_pl);
pl.print();
```
Output:
```
Time Stepping ->
Final Time = 1
Time Step = 0.001
Number of Time Steps = 1000 [default]
Equations ->
Laguerre 1 ->
Order = 3 [unused]
Laguerre 2 ->
Order = 5 [unused]
Hermite 1 ->
Order = 0 [unused]
```
## Definition of Done
<!---
Tell us what needs to happen. If necessary, give us a task list along the
lines of:
- [ ] First do this.
- [ ] Then do that.
- [ ] Also this other thing.
-->
- [x] Unit tests added for all additional modification and reconciliation methods
- [x] Documentation added with usage examples as well
- [ ] Updates to tutorials?
## Possible Solution
<!---
Not obligatory, but suggest a fix for the bug or documentation, or suggest
ideas on how to implement the addition or change.
-->
A merge request will be attached to this issue with an initial implementation, unit tests, and documentation.
## Steps to Reproduce
<!---
Provide a link to a live example, or an unambiguous set of steps to reproduce
this issue. Include code to reproduce, if relevant.
1. Do this.
1. Do that.
1. Shake fist angrily at computer.
-->
## Your Environment
<!---
Include relevant details about your environment such that we can replicate this
issue.
-->
- **Relevant repo SHA1s:**
- **Relevant configure flags or configure script:**
- **Operating system and version:**
- **Compiler and TPL versions:**
## Related Issues
<!---
If applicable, let us know how this bug is related to any other open issues:
-->
* Blocks
* Is blocked by
* Follows
* Precedes
* Related to
* Part of
* Composed of
## Additional Information
<!---
Anything else that might be helpful for us to know in addressing this issue:
* Configure log file:
* Build log file:
* Test log file:
* When was the last time everything worked (date/time; SHA1s; etc.)?
* What did you do that made the bug rear its ugly head?
* Have you tried turning it off and on again?
-->
https://gitlab.osti.gov/jmwille/Trilinos/-/issues/4790TpetraCore tests failing in ATDM sems-rhel7+cuda+complex build2019-04-20T20:21:19ZJames WillenbringTpetraCore tests failing in ATDM sems-rhel7+cuda+complex build*Created by: fryeguy52*
CC: @trilinos/tpetra, @kddevin (Trilinos Data Services Product Lead), @bartlettroscoe, @fryeguy52
<Checklist>
<???: Add label "ATDM">
<???: Add label "bug"?>
<???: Add label for affected packages (e.g. "Mu...*Created by: fryeguy52*
CC: @trilinos/tpetra, @kddevin (Trilinos Data Services Product Lead), @bartlettroscoe, @fryeguy52
<Checklist>
<???: Add label "ATDM">
<???: Add label "bug"?>
<???: Add label for affected packages (e.g. "MueLu", "Tpetra", "Kokkos", etc.)>
<???: Add milestone "Initial cleanup of new ATDM builds of Trilinos" or "Keep promoted ATDM builds of Trilinos clean">
<???: Once GitHub Issue is created, add entries for tests to TrilinosATDMStatus/*.csv files>
<???: Add label "PA: ???Project Area???" (e.g. "PA: Linear Solvers", "PA: Data Services")>
## Next Action Status
<status-and-or-first-action>
## Description
As shown in [this query](https://testing.sandia.gov/cdash/queryTests.php?project=Trilinos&filtercombine=and&filtercombine=&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercount=6&showfilters=1&filtercombine=and&field1=buildname&compare1=61&value1=Trilinos-atdm-sems-rhel7-cuda-9.2-Volta70-complex-static-release-debug&field2=testname&compare2=65&value2=TpetraCore&field3=site&compare3=61&value3=sems-rhel7&field4=buildstarttime&compare4=84&value4=2019-04-02T00%3A00%3A00&field5=buildstarttime&compare5=83&value5=2019-03-25T00%3A00%3A00&field6=status&compare6=61&value6=Failed) the build:
* Trilinos-atdm-sems-rhel7-cuda-9.2-Volta70-complex-static-release-debug
has several failing TpetraCore tests. The following are failing everyday ([CDash](https://testing.sandia.gov/cdash/queryTests.php?project=Trilinos&filtercombine=and&filtercombine=&filtercombine=and&filtercombine=and&filtercombine=and&filtercount=6&showfilters=1&filtercombine=and&field1=buildname&compare1=61&value1=Trilinos-atdm-sems-rhel7-cuda-9.2-Volta70-complex-static-release-debug&field2=testname&compare2=65&value2=TpetraCore_gemm_m_eq_&field3=testname&compare3=66&value3=_MPI_1&field4=site&compare4=61&value4=sems-rhel7&field5=buildstarttime&compare5=84&value5=2019-04-02T00%3A00%3A00&field6=buildstarttime&compare6=83&value6=2019-03-03T00%3A00%3A00))
* TpetraCore_gemm_m_eq_1_MPI_1
* TpetraCore_gemm_m_eq_1_MPI_2
* TpetraCore_gemm_m_eq_1_MPI_5
* TpetraCore_gemm_m_eq_1_MPI_13
While these have failed randomly over the last couple weeks ([CDash](https://testing.sandia.gov/cdash/queryTests.php?project=Trilinos&filtercombine=and&filtercombine=&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercount=7&showfilters=1&filtercombine=and&field1=buildname&compare1=61&value1=Trilinos-atdm-sems-rhel7-cuda-9.2-Volta70-complex-static-release-debug&field2=testname&compare2=65&value2=TpetraCore&field3=testname&compare3=64&value3=_gemm_m_eq_&field4=site&compare4=61&value4=sems-rhel7&field5=buildstarttime&compare5=84&value5=2019-04-02T00%3A00%3A00&field6=buildstarttime&compare6=83&value6=2019-03-19T00%3A00%3A00&field7=status&compare7=61&value7=Failed)):
* TpetraCore_MultiVector_MicroBenchmark_MPI_1
* TpetraCore_Map_Bug5822_2_MPI_2
* TpetraCore_getEntryOnHost_MPI_1
* TpetraCore_gemv_MPI_1
* TpetraCore_deep_copy_MultiVector_to_SerialDenseMatrix_MPI_1
* TpetraCore_createMirrorView_MPI_1
several more tests have failed over the last 2 weeks in similar complex shared build shown [here](https://testing.sandia.gov/cdash/queryTests.php?project=Trilinos&filtercombine=and&filtercombine=&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercount=6&showfilters=1&filtercombine=and&field1=buildname&compare1=61&value1=Trilinos-atdm-sems-rhel7-cuda-9.2-Volta70-complex-shared-release-debug&field2=testname&compare2=65&value2=TpetraCore&field3=site&compare3=61&value3=sems-rhel7&field4=buildstarttime&compare4=84&value4=today&field5=buildstarttime&compare5=83&value5=2%20weeks%20ago&field6=status&compare6=61&value6=failed):
* Trilinos-atdm-sems-rhel7-cuda-9.2-Volta70-complex-shared-release-debug
Those tests are:
* TpetraCore_MatrixMatrix_UnitTests_MPI_4
* TpetraCore_Issue_607_MPI_4
* TpetraCore_Issue_114_MPI_4
* TpetraCore_Issue601_MPI_4
* TpetraCore_ImportExport2_UnitTests_MPI_4
* TpetraCore_ImportBug5430_MPI_4
* TpetraCore_Import_Union_MPI_4
* TpetraCore_CrsMatrix_NonlocalSumInto_Ignore_MPI_4
* TpetraCore_CrsMatrix_gaussSeidel_MPI_4
* TpetraCore_CrsGraph_getNumDiags_MPI_4
* TpetraCore_AddProfiling_UnitTests_MPI_4
<more-details>
## Current Status on CDash
[Failed TpetraCore tests for the current testing day](https://testing.sandia.gov/cdash/queryTests.php?project=Trilinos&filtercombine=and&filtercombine=&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercount=6&showfilters=1&filtercombine=and&field1=buildname&compare1=61&value1=Trilinos-atdm-sems-rhel7-cuda-9.2-Volta70-complex-static-release-debug&field2=testname&compare2=65&value2=TpetraCore&field3=site&compare3=61&value3=sems-rhel7&field4=buildstarttime&compare4=84&value4=today&field5=buildstarttime&compare5=83&value5=yesterday&field6=status&compare6=61&value6=failed)
## Steps to Reproduce
One should be able to reproduce this failure on with a sems rhel6 environment as described in:
* https://github.com/trilinos/Trilinos/blob/develop/cmake/std/atdm/README.md
More specifically, the commands given for with a sems rhel6 environment are provided at:
* https://github.com/trilinos/Trilinos/blob/develop/cmake/std/atdm/README.md#sems-rhel6-environment
The exact commands to reproduce this issue should be:
```
$ cd <some_build_dir>/
$ source $TRILINOS_DIR/cmake/std/atdm/load-env.sh Trilinos-atdm-sems-rhel7-cuda-9.2-Volta70-complex-static-release-debug
$ cmake \
-GNinja \
-DTrilinos_CONFIGURE_OPTIONS_FILE:STRING=cmake/std/atdm/ATDMDevEnv.cmake \
-DTrilinos_ENABLE_TESTS=ON -DTrilinos_ENABLE_Tpetra=ON \
$TRILINOS_DIR
$ make NP=16
$ ctest -j8
```
Initial cleanup of new ATDM builds of Trilinoshttps://gitlab.osti.gov/jmwille/Trilinos/-/issues/4789Unused Tpetra var warning in header breaking SPARC Trilinos Integration build...2019-04-05T13:42:55ZJames WillenbringUnused Tpetra var warning in header breaking SPARC Trilinos Integration builds with -Werror*Created by: bartlettroscoe*
CC: @trilinos/tpetra, @kddevin (Trilinos Data Services Product Lead), @bartlettroscoe, @fryeguy52, @mhoemmen
## Next Action Status
<status-and-or-first-action>
## Description
Recently Tpetra wa...*Created by: bartlettroscoe*
CC: @trilinos/tpetra, @kddevin (Trilinos Data Services Product Lead), @bartlettroscoe, @fryeguy52, @mhoemmen
## Next Action Status
<status-and-or-first-action>
## Description
Recently Tpetra was updated to add a new unused var warning:
```
.../Tpetra_Details_StaticView.hpp:168:9: error: unused type alias 'host_device_type' [-Werror,-Wunused-local-typedef]
```
This is bringing down the [SPARC Trilinos Integration builds](https://snl-wiki.sandia.gov/display/CoodinatedDevOpsATDM/SPARC+Trilinos+Integration+Process) for Clang and GCC due to the SPARC setting `-Werror` in those builds. Therefore, we get no build or test results in those builds due to this.
## Current Status on CDash
The current status can only be seen in the SPARC Trilinos Integration builds shown over the last few days [here](http://compsim-dashboard.sandia.gov/cdash/index.php?project=SPARC&date=2019-03-30&filtercombine=and&filtercount=2&showfilters=1&filtercombine=and&field1=buildname&compare1=66&value1=-trildev&field2=buildstarttime&compare2=83&value2=5%20days%20ago).
## Steps to Reproduce
Instructions to reproduce builds are at:
* https://github.com/trilinos/Trilinos/blob/develop/cmake/std/atdm/README.md
More specifically, the commands given for the system 'cee-rhel6' are provided at:
* https://github.com/trilinos/Trilinos/blob/develop/cmake/std/atdm/README.md#cee-rhel6-environment
The exact commands to reproduce this warning should (hopefully) be:
```
$ cd <some_build_dir>/
$ source $TRILINOS_DIR/cmake/std/atdm/load-env.sh \
Trilinos-atdm-cee-rhel6_clang-5.0.1_openmpi-1.10.2_serial_static_opt
$ cmake \
-GNinja \
-DTrilinos_CONFIGURE_OPTIONS_FILE:STRING=cmake/std/atdm/ATDMDevEnv.cmake \
-DTrilinos_ENABLE_TESTS=ON -DTrilinos_ENABLE_Tpetra=ON \
$TRILINOS_DIR
$ make NP=16
```