Trilinos issueshttps://gitlab.osti.gov/jmwille/Trilinos/-/issues2019-06-08T15:27:25Zhttps://gitlab.osti.gov/jmwille/Trilinos/-/issues/5104Pamgen: Remove Build Warnings2019-06-08T15:27:25ZJames WillenbringPamgen: Remove Build Warnings*Created by: ZUUL42*
## Enhancement
@trilinos/pamgen
Issue #3178 is working toward turning Warnings as Errors on for _all_ packages in order in ensure Trilinos maintains a high level of SQA practices across the project.
Currently ...*Created by: ZUUL42*
## Enhancement
@trilinos/pamgen
Issue #3178 is working toward turning Warnings as Errors on for _all_ packages in order in ensure Trilinos maintains a high level of SQA practices across the project.
Currently Pamgen has a warning that need to be handled before we can set Werror for Pamgen and eventually all packages.
A recent test build was performed with -Werror set. [The CDash report can be found here.](https://testing.sandia.gov/cdash/index.php?project=Trilinos&parentid=4989737&filtercount=2&showfilters=1&field1=buildstamp&compare1=63&value1=Experimental&field2=buildstarttime&compare2=83&value2=2019/04/30&filtercombine=and)
Once the Pamgen build doesn't emit any warnings that will be promoted to errors, [we can set `-Werror` in the GCC 7.2.0 automated build](https://github.com/ZUUL42/Trilinos/tree/Werror_Pamgen).https://gitlab.osti.gov/jmwille/Trilinos/-/issues/4105Closing old PR #1019 "Misleading indentation (whitespace only)" 2018-12-20T18:57:01ZJames WillenbringClosing old PR #1019 "Misleading indentation (whitespace only)" *Created by: william76*
Pull Request #1019 "Misleading indentation (whitespace only)" is really old and appears to be stale and/or abandoned. I'm closing that PR and creating this issue to link to it.
I'm doing this because we need...*Created by: william76*
Pull Request #1019 "Misleading indentation (whitespace only)" is really old and appears to be stale and/or abandoned. I'm closing that PR and creating this issue to link to it.
I'm doing this because we need to close out some of the old PR's due to some GitHub limitations on the number of checks/hour that are allowed. The pull request autotester uses a polling model to check existing pull requests' status flags, etc. and we have occasionally hit that limit, which causes GitHub to reject the queries and can cause the Autotester to fail on a PR. If this PR needs to be brought back to life it can easily be reopened on the pull request page.
If this PR is truly dead, please close out this issue ticket.
FYI: @nschloe @egboman
PR #1019 touches files in these packages:
@trilinos/epetra
@trilinos/aztecoo
@trilinos/epetraext
@trilinos/isorropia
@trilinos/ml
@trilinos/pamgen https://gitlab.osti.gov/jmwille/Trilinos/-/issues/2221Pamgen,kokkos-kernels: NVCC build error with -D CMAKE_CXX_USE_RESPONSE_FILE_F...2019-03-05T22:14:10ZJames WillenbringPamgen,kokkos-kernels: NVCC build error with -D CMAKE_CXX_USE_RESPONSE_FILE_FOR_OBJECTS=ON*Created by: mhoemmen*
kokkos-kernels requires `-D CMAKE_CXX_USE_RESPONSE_FILE_FOR_OBJECTS=ON` for CUDA builds, even with complex arithmetic disabled.
https://github.com/trilinos/Trilinos/issues/2115#issuecomment-357750048
However...*Created by: mhoemmen*
kokkos-kernels requires `-D CMAKE_CXX_USE_RESPONSE_FILE_FOR_OBJECTS=ON` for CUDA builds, even with complex arithmetic disabled.
https://github.com/trilinos/Trilinos/issues/2115#issuecomment-357750048
However, Pamgen does not build, given those settings.
@trilinos/kokkos-kernels @trilinos/pamgen @trilinos/framework
CC: @rrdrake @prwolfe
## Expectations
1. Whatever flags kokkos-kernels needs to build on CUDA, they need not to block building other essential packages.
2. kokkos-kernels needs to document its CMake requirements.
3. kokkos-kernels needs to fail at configure time, with an informative message, if the CMake variables that it needs are not set.
## Current Behavior
```
$ make
[ 0%] Linking CXX shared library libpamgen.so
nvcc fatal : No input files specified; use option --help for more information
make[2]: *** [packages/pamgen/src/libpamgen.so.12.13] Error 1
make[1]: *** [packages/pamgen/src/CMakeFiles/pamgen.dir/all] Error 2
make: *** [all] Error 2
```
## Motivation and Context
Many downstream tests, including Belos and Ifpack2, depend on Pamgen. This blocks adequate Trilinos testing on CUDA.
## Steps to Reproduce
```
$ module list
Currently Loaded Modulefiles:
1) sems-env 3) sems-cmake/3.3.2 5) kokkos-cuda/8.0.44
2) kokkos-env 4) sems-gcc/5.3.0 6) kokkos-openmpi/2.0.1/cuda
```
CMake configuration:
```
-D Trilinos_ENABLE_EXPLICIT_INSTANTIATION:BOOL=ON
-D BUILD_SHARED_LIBS:BOOL=ON
-D Trilinos_ENABLE_OpenMP:BOOL=ON
-D Kokkos_ENABLE_OpenMP:BOOL=ON
-D Tpetra_INST_OPENMP:BOOL=ON
-D Trilinos_SHOW_DEPRECATED_WARNINGS:BOOL=ON
-D Trilinos_ENABLE_Fortran:BOOL=OFF
-D TPL_ENABLE_CUDA:BOOL=ON
-D KOKKOS_ARCH="SNB;Kepler35"
-D Kokkos_ENABLE_Cuda:BOOL=ON
-D Kokkos_ENABLE_Cuda_UVM:BOOL=ON
-D Tpetra_INST_CUDA:BOOL=ON
-D Kokkos_ENABLE_Cuda_Lambda:BOOL=ON
-D CMAKE_CXX_USE_RESPONSE_FILE_FOR_OBJECTS:BOOL=ON
-D CMAKE_CXX_FLAGS:STRING="-Wall"
-D Trilinos_CXX11_FLAGS:STRING="-std=c++11 --expt-extended-lambda"
-D TPL_ENABLE_MKL:BOOL=OFF
-D TPL_ENABLE_Matio:BOOL=OFF
-D TPL_ENABLE_SuperLU:BOOL=OFF
-D TPL_ENABLE_Zlib:BOOL=OFF
-D TPL_ENABLE_Netcdf:BOOL=OFF
-D TPL_ENABLE_HDF5:BOOL=OFF
-D TPL_ENABLE_ParMETIS:BOOL=OFF
-D TPL_ENABLE_Boost:BOOL=OFF
-D TPL_ENABLE_BoostLib:BOOL=OFF
-D TPL_ENABLE_yaml-cpp:BOOL=OFF
-D TPL_ENABLE_MPI:BOOL=ON
```
## Your Environment
- develop branch, commit 400765e21e17dfb995e0f4a2759ce9c5f961b685 (likely not related)
## Related Issues
* Blocks #2115 https://gitlab.osti.gov/jmwille/Trilinos/-/issues/1945several packages call exit() in the library2017-11-07T18:25:18ZJames Willenbringseveral packages call exit() in the library*Created by: nschloe*
Discussions of why calling `exit()` in production code go back as far as 2010; cf. https://software.sandia.gov/bugzilla/show_bug.cgi?id=4969. Unfortunately, many libraries still call `exit()`:
```
$ lintian * | g...*Created by: nschloe*
Discussions of why calling `exit()` in production code go back as far as 2010; cf. https://software.sandia.gov/bugzilla/show_bug.cgi?id=4969. Unfortunately, many libraries still call `exit()`:
```
$ lintian * | grep shlib-calls-exit
X: libtrilinos-zoltan12: shlib-calls-exit usr/lib/x86_64-linux-gnu/libtrilinos_zoltan.so.12.12.1
X: libtrilinos-aztecoo12: shlib-calls-exit usr/lib/x86_64-linux-gnu/libtrilinos_aztecoo.so.12.12.1
X: libtrilinos-muelu12: shlib-calls-exit usr/lib/x86_64-linux-gnu/libtrilinos_muelu.so.12.12.1
X: libtrilinos-nox12: shlib-calls-exit usr/lib/x86_64-linux-gnu/libtrilinos_noxepetra.so.12.12.1
X: libtrilinos-stokhos12: shlib-calls-exit usr/lib/x86_64-linux-gnu/libtrilinos_stokhos_muelu.so.12.12.1
X: libtrilinos-galeri12: shlib-calls-exit usr/lib/x86_64-linux-gnu/libtrilinos_galeri-epetra.so.12.12.1
X: libtrilinos-epetraext12: shlib-calls-exit usr/lib/x86_64-linux-gnu/libtrilinos_epetraext.so.12.12.1
X: libtrilinos-shylu12: shlib-calls-exit usr/lib/x86_64-linux-gnu/libtrilinos_shylu.so.12.12.1
X: libtrilinos-pamgen12: shlib-calls-exit usr/lib/x86_64-linux-gnu/libtrilinos_pamgen.so.12.12.1
X: libtrilinos-ml12: shlib-calls-exit usr/lib/x86_64-linux-gnu/libtrilinos_ml.so.12.12.1
X: libtrilinos-triutils12: shlib-calls-exit usr/lib/x86_64-linux-gnu/libtrilinos_triutils.so.12.12.1
X: libtrilinos-ifpack12: shlib-calls-exit usr/lib/x86_64-linux-gnu/libtrilinos_ifpack.so.12.12.1
X: libtrilinos-pliris12: shlib-calls-exit usr/lib/x86_64-linux-gnu/libtrilinos_dpliris.so.12.12.1
X: libtrilinos-trilinoscouplings12: shlib-calls-exit usr/lib/x86_64-linux-gnu/libtrilinos_trilinoscouplings.so.12.12.1
```
Would be great to see some progress here.https://gitlab.osti.gov/jmwille/Trilinos/-/issues/1422Get all SERIAL tests in Trilinos to run as 1-process tests in MPI builds2017-08-22T19:15:01ZJames WillenbringGet all SERIAL tests in Trilinos to run as 1-process tests in MPI builds*Created by: bartlettroscoe*
**CC:** @trilinos/framework
**Description:**
A big problem we have with SQA and test of Trilinos is that many Trilinos packages have significant numbers of tests that only run in a non-MPI serial buil...*Created by: bartlettroscoe*
**CC:** @trilinos/framework
**Description:**
A big problem we have with SQA and test of Trilinos is that many Trilinos packages have significant numbers of tests that only run in a non-MPI serial build (e.g. Pamgan #1415 and SEACAS #1392). That means that running the standard CI build MPI_RELEASE_DEBUG_SHARED_PT does not run any of these tests.
This story is to create a place-marker for package-specific stories to get all of their single-process tests now running only in the non-MPI serial build to also run in the MPI builds. This way, the set of tests run in a serial build will be a strict subset of the tests run in the MPI build. Therefore, passing the MPI build will give higher confidence that the serial build will also pass all of its tests.
Improve productivity, stability, and quality of Trilinoshttps://gitlab.osti.gov/jmwille/Trilinos/-/issues/788Pamgen build error with GCC 4.4.7 when C++11 is disabled2016-11-07T20:41:55ZJames WillenbringPamgen build error with GCC 4.4.7 when C++11 is disabled*Created by: mhoemmen*
@csiefer2 is Pamgen supposed to be able to build without C++11 support?
See #785 for reported build errors (not the Teuchos errors -- scroll down to the bottom).*Created by: mhoemmen*
@csiefer2 is Pamgen supposed to be able to build without C++11 support?
See #785 for reported build errors (not the Teuchos errors -- scroll down to the bottom).https://gitlab.osti.gov/jmwille/Trilinos/-/issues/23Pamgen: tests 1, 20, 21 failing2016-06-30T00:16:25ZJames WillenbringPamgen: tests 1, 20, 21 failing*Created by: nschloe*
On a clean Pamgen build,
```
cmake \
-DTrilinos_ENABLE_Pamgen:BOOL=ON \
-DTrilinos_ENABLE_TESTS:BOOL=ON \
-DBUILD_SHARED_LIBS:BOOL=ON \
../../source-upstream/
```
tests 1, 20, and 21 are failing, the firs...*Created by: nschloe*
On a clean Pamgen build,
```
cmake \
-DTrilinos_ENABLE_Pamgen:BOOL=ON \
-DTrilinos_ENABLE_TESTS:BOOL=ON \
-DBUILD_SHARED_LIBS:BOOL=ON \
../../source-upstream/
```
tests 1, 20, and 21 are failing, the first with a timeout:
```
ctest
Test project /home/nschloe/software/trilinos/build/pamgen
Start 1: Pamgen_run_test_script_l_01
1/69 Test #1: Pamgen_run_test_script_l_01 ......***Timeout 1500.02 sec
Start 2: Pamgen_run_test_script_01
2/69 Test #2: Pamgen_run_test_script_01 ........ Passed 0.21 sec
[...]
19/69 Test #19: Pamgen_run_test_script_19 ........ Passed 0.24 sec
Start 20: Pamgen_run_test_script_20
*** buffer overflow detected ***: /home/nschloe/software/trilinos/build/pamgen/packages/pamgen/test/pamgen_test_driver.exe terminated
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x7338f)[0x7f82de13e38f]
/lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x5c)[0x7f82de1d5c9c]
/lib/x86_64-linux-gnu/libc.so.6(+0x109b60)[0x7f82de1d4b60]
/lib/x86_64-linux-gnu/libc.so.6(+0x109069)[0x7f82de1d4069]
/lib/x86_64-linux-gnu/libc.so.6(_IO_default_xsputn+0xbc)[0x7f82de14670c]
/lib/x86_64-linux-gnu/libc.so.6(_IO_vfprintf+0x7fa)[0x7f82de1154ea]
/lib/x86_64-linux-gnu/libc.so.6(__vsprintf_chk+0x84)[0x7f82de1d40f4]
/lib/x86_64-linux-gnu/libc.so.6(__sprintf_chk+0x7d)[0x7f82de1d404d]
/home/nschloe/software/trilinos/build/pamgen/packages/pamgen/test/pamgen_test_driver.exe(main+0x317)[0x41d7e7]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x7f82de0ecec5]
/home/nschloe/software/trilinos/build/pamgen/packages/pamgen/test/pamgen_test_driver.exe[0x41e04f]
======= Memory map: ========
00400000-004b6000 r-xp 00000000 08:04 17721233 /home/nschloe/software/trilinos/build/pamgen/packages/pamgen/test/pamgen_test_driver.exe
006b5000-006b6000 r--p 000b5000 08:04 17721233 /home/nschloe/software/trilinos/build/pamgen/packages/pamgen/test/pamgen_test_driver.exe
006b6000-006b7000 rw-p 000b6000 08:04 17721233 /home/nschloe/software/trilinos/build/pamgen/packages/pamgen/test/pamgen_test_driver.exe
006b7000-006b9000 rw-p 00000000 00:00 0
01911000-01932000 rw-p 00000000 00:00 0 [heap]
7f82de0cb000-7f82de286000 r-xp 00000000 08:02 1835132 /lib/x86_64-linux-gnu/libc-2.19.so
7f82de286000-7f82de485000 ---p 001bb000 08:02 1835132 /lib/x86_64-linux-gnu/libc-2.19.so
7f82de485000-7f82de489000 r--p 001ba000 08:02 1835132 /lib/x86_64-linux-gnu/libc-2.19.so
7f82de489000-7f82de48b000 rw-p 001be000 08:02 1835132 /lib/x86_64-linux-gnu/libc-2.19.so
7f82de48b000-7f82de490000 rw-p 00000000 00:00 0
7f82de490000-7f82de4a6000 r-xp 00000000 08:02 1852617 /lib/x86_64-linux-gnu/libgcc_s.so.1
7f82de4a6000-7f82de6a5000 ---p 00016000 08:02 1852617 /lib/x86_64-linux-gnu/libgcc_s.so.1
7f82de6a5000-7f82de6a6000 rw-p 00015000 08:02 1852617 /lib/x86_64-linux-gnu/libgcc_s.so.1
7f82de6a6000-7f82de7ab000 r-xp 00000000 08:02 1835112 /lib/x86_64-linux-gnu/libm-2.19.so
7f82de7ab000-7f82de9aa000 ---p 00105000 08:02 1835112 /lib/x86_64-linux-gnu/libm-2.19.so
7f82de9aa000-7f82de9ab000 r--p 00104000 08:02 1835112 /lib/x86_64-linux-gnu/libm-2.19.so
7f82de9ab000-7f82de9ac000 rw-p 00105000 08:02 1835112 /lib/x86_64-linux-gnu/libm-2.19.so
7f82de9ac000-7f82dea92000 r-xp 00000000 08:02 2251778 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19
7f82dea92000-7f82dec91000 ---p 000e6000 08:02 2251778 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19
7f82dec91000-7f82dec99000 r--p 000e5000 08:02 2251778 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19
7f82dec99000-7f82dec9b000 rw-p 000ed000 08:02 2251778 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19
7f82dec9b000-7f82decb0000 rw-p 00000000 00:00 0
7f82decb0000-7f82decd3000 r-xp 00000000 08:02 1835129 /lib/x86_64-linux-gnu/ld-2.19.so
7f82dee96000-7f82dee9b000 rw-p 00000000 00:00 0
7f82deecf000-7f82deed2000 rw-p 00000000 00:00 0
7f82deed2000-7f82deed3000 r--p 00022000 08:02 1835129 /lib/x86_64-linux-gnu/ld-2.19.so
7f82deed3000-7f82deed4000 rw-p 00023000 08:02 1835129 /lib/x86_64-linux-gnu/ld-2.19.so
7f82deed4000-7f82deed5000 rw-p 00000000 00:00 0
7ffcdb242000-7ffcdb263000 rw-p 00000000 00:00 0 [stack]
7ffcdb358000-7ffcdb35a000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
20/69 Test #20: Pamgen_run_test_script_20 ........***Failed Required regular expression not found.Regex=[OVERALL FINAL RESULT: TEST PASSED .Pamgen_run_test_script_20.
] 0.69 sec
Start 21: Pamgen_run_test_script_21
*** buffer overflow detected ***: /home/nschloe/software/trilinos/build/pamgen/packages/pamgen/test/pamgen_test_driver.exe terminated
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x7338f)[0x7f12a88de38f]
/lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x5c)[0x7f12a8975c9c]
/lib/x86_64-linux-gnu/libc.so.6(+0x109b60)[0x7f12a8974b60]
/lib/x86_64-linux-gnu/libc.so.6(+0x109069)[0x7f12a8974069]
/lib/x86_64-linux-gnu/libc.so.6(_IO_default_xsputn+0xbc)[0x7f12a88e670c]
/lib/x86_64-linux-gnu/libc.so.6(_IO_vfprintf+0x7fa)[0x7f12a88b54ea]
/lib/x86_64-linux-gnu/libc.so.6(__vsprintf_chk+0x84)[0x7f12a89740f4]
/lib/x86_64-linux-gnu/libc.so.6(__sprintf_chk+0x7d)[0x7f12a897404d]
/home/nschloe/software/trilinos/build/pamgen/packages/pamgen/test/pamgen_test_driver.exe(main+0x317)[0x41d7e7]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x7f12a888cec5]
/home/nschloe/software/trilinos/build/pamgen/packages/pamgen/test/pamgen_test_driver.exe[0x41e04f]
======= Memory map: ========
00400000-004b6000 r-xp 00000000 08:04 17721233 /home/nschloe/software/trilinos/build/pamgen/packages/pamgen/test/pamgen_test_driver.exe
006b5000-006b6000 r--p 000b5000 08:04 17721233 /home/nschloe/software/trilinos/build/pamgen/packages/pamgen/test/pamgen_test_driver.exe
006b6000-006b7000 rw-p 000b6000 08:04 17721233 /home/nschloe/software/trilinos/build/pamgen/packages/pamgen/test/pamgen_test_driver.exe
006b7000-006b9000 rw-p 00000000 00:00 0
00862000-00883000 rw-p 00000000 00:00 0 [heap]
7f12a886b000-7f12a8a26000 r-xp 00000000 08:02 1835132 /lib/x86_64-linux-gnu/libc-2.19.so
7f12a8a26000-7f12a8c25000 ---p 001bb000 08:02 1835132 /lib/x86_64-linux-gnu/libc-2.19.so
7f12a8c25000-7f12a8c29000 r--p 001ba000 08:02 1835132 /lib/x86_64-linux-gnu/libc-2.19.so
7f12a8c29000-7f12a8c2b000 rw-p 001be000 08:02 1835132 /lib/x86_64-linux-gnu/libc-2.19.so
7f12a8c2b000-7f12a8c30000 rw-p 00000000 00:00 0
7f12a8c30000-7f12a8c46000 r-xp 00000000 08:02 1852617 /lib/x86_64-linux-gnu/libgcc_s.so.1
7f12a8c46000-7f12a8e45000 ---p 00016000 08:02 1852617 /lib/x86_64-linux-gnu/libgcc_s.so.1
7f12a8e45000-7f12a8e46000 rw-p 00015000 08:02 1852617 /lib/x86_64-linux-gnu/libgcc_s.so.1
7f12a8e46000-7f12a8f4b000 r-xp 00000000 08:02 1835112 /lib/x86_64-linux-gnu/libm-2.19.so
7f12a8f4b000-7f12a914a000 ---p 00105000 08:02 1835112 /lib/x86_64-linux-gnu/libm-2.19.so
7f12a914a000-7f12a914b000 r--p 00104000 08:02 1835112 /lib/x86_64-linux-gnu/libm-2.19.so
7f12a914b000-7f12a914c000 rw-p 00105000 08:02 1835112 /lib/x86_64-linux-gnu/libm-2.19.so
7f12a914c000-7f12a9232000 r-xp 00000000 08:02 2251778 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19
7f12a9232000-7f12a9431000 ---p 000e6000 08:02 2251778 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19
7f12a9431000-7f12a9439000 r--p 000e5000 08:02 2251778 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19
7f12a9439000-7f12a943b000 rw-p 000ed000 08:02 2251778 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19
7f12a943b000-7f12a9450000 rw-p 00000000 00:00 0
7f12a9450000-7f12a9473000 r-xp 00000000 08:02 1835129 /lib/x86_64-linux-gnu/ld-2.19.so
7f12a9636000-7f12a963b000 rw-p 00000000 00:00 0
7f12a966f000-7f12a9672000 rw-p 00000000 00:00 0
7f12a9672000-7f12a9673000 r--p 00022000 08:02 1835129 /lib/x86_64-linux-gnu/ld-2.19.so
7f12a9673000-7f12a9674000 rw-p 00023000 08:02 1835129 /lib/x86_64-linux-gnu/ld-2.19.so
7f12a9674000-7f12a9675000 rw-p 00000000 00:00 0
7fff4de72000-7fff4de93000 rw-p 00000000 00:00 0 [stack]
7fff4dfad000-7fff4dfaf000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
21/69 Test #21: Pamgen_run_test_script_21 ........***Failed Required regular expression not found.Regex=[OVERALL FINAL RESULT: TEST PASSED .Pamgen_run_test_script_21.
] 0.21 sec
Start 22: Pamgen_run_test_script_22
22/69 Test #22: Pamgen_run_test_script_22 ........ Passed 0.23 sec
[...]
Start 69: Pamgen_run_test_script_69
69/69 Test #69: Pamgen_run_test_script_69 ........ Passed 0.04 sec
96% tests passed, 3 tests failed out of 69
Label Time Summary:
Pamgen = 1510.36 sec
Total Test time (real) = 1510.91 sec
The following tests FAILED:
1 - Pamgen_run_test_script_l_01 (Timeout)
20 - Pamgen_run_test_script_20 (Failed)
21 - Pamgen_run_test_script_21 (Failed)
Errors while running CTest
```