Skip to content

Selective disables of a few individual unit tests in Kokkos and KokkosKernels for cuda-debug build on 'white' and 'ride'

James Willenbring requested to merge bartlettroscoe:2827-disable-kokkos-kokkos-kernels-unit-tests into develop

Created by: bartlettroscoe

CC: @trilinos/kokkos, @trilinos/kokkos-kernels, @fryeguy52

Description

This PR disables a couple of the more expensive individual unit tests for a Kokkos and KokkosKernels test for just the cuda-debug build on just 'white'/'ride'. This addresses the last of the errors related to #2827 (closed).

As part of this, I extended TriBITS to support a _EXTRA_ARGS cache var that allows you to append arbitrary command-line arguments to specific tests added with TRIBITS_ADD_TEST(). I used this to add the --gtest_filter arguments to disable the specific unit tests.

I also included a commit fixing some typos in a README.md file.

Motivation and Context

These tests were timing out at over 10 minutes for this build.

How Has This Been Tested?

I tested this on white with:

$ bsub -x -Is -q rhel7F -n 16 \
  ./checkin-test-atdm.sh cuda-debug --enable-packages=Kokkos,KokkosKernels --local-do-all

and it returned:

...
31/35 Test #29: KokkosKernels_sparse_cuda_MPI_1 ..................   Passed  235.92 sec
32/35 Test  #2: KokkosCore_UnitTest_Cuda_MPI_1 ...................   Passed  241.43 sec
33/35 Test #25: KokkosContainers_UnitTest_Serial_MPI_1 ...........   Passed  263.52 sec
34/35 Test #33: KokkosKernels_sparse_serial_MPI_1 ................   Passed  358.63 sec
35/35 Test #34: KokkosKernels_graph_serial_MPI_1 .................   Passed  408.13 sec

100% tests passed, 0 tests failed out of 35

Subproject Time Summary:
Kokkos           = 814.77 sec*proc (27 tests)
KokkosKernels    = 1282.89 sec*proc (8 tests)

Total Test time (real) = 408.15 sec

So that is now under the 600 sec (10 minute) time limit.

Before this change, these two tests timed out with:

...
34/35 Test #33: KokkosKernels_sparse_serial_MPI_1 ................***Timeout 600.07 sec
35/35 Test #25: KokkosContainers_UnitTest_Serial_MPI_1 ...........***Timeout 600.08 sec

94% tests passed, 2 tests failed out of 35

Subproject Time Summary:
Kokkos           = 1145.23 sec*proc (27 tests)
KokkosKernels    = 1516.06 sec*proc (8 tests)

Total Test time (real) = 600.12 sec

The following tests FAILED:
	 25 - KokkosContainers_UnitTest_Serial_MPI_1 (Timeout)
	 33 - KokkosKernels_sparse_serial_MPI_1 (Timeout)
Errors while running CTest

Checklist

  • My commit messages mention the appropriate GitHub issue numbers.
  • All new and existing tests passed.

Merge request reports