GitLab

Created by: jjellio

I am having a problem configuring Trilinos using the Cray programming environment.

Specifically, I have ParMETIS and Zoltan2 enabled. Zoltan2 requires ParMETIS >= 4.0.3.

The test for this builds a simple binary that includes parmetis.h and checks that the version is sufficient.

The problem seems to be that on Mutrino, the node used for compiling is not the same ARCH as the target. E.g., Cray's CC sets -xCORE-AVX2 for haswell, but the login nodes actually support AVX1 only.

The offending configure step is part of the Find ParMetis macro:

-- Performing Test HAVE_PARMETIS_VERSION_4_0_3 -- Run Output: Please verify that both the operating system and the processor support Intel(R) MOVBE, F16C, FMA, BMI, LZCNT and AVX2 instructions.

In this case, the test that fails is: cmake/TPLs/FindTPLParMETIS.cmake, which calls CHECK_C_SOURCE_RUNS. The test executable is built using the cray wrappers (CC/cc) and will not run because the system does not support the instruction set.

FUNCTION(CHECK_PARMETIS_HAS_VERSION_4_0_3  VARNAME)
  SET(SOURCE
  "
  #include <stdio.h>
  #include <parmetis.h>
  int main()
  {
    if( PARMETIS_MAJOR_VERSION > 4 )
      return 0;
    if( PARMETIS_MAJOR_VERSION == 4 )
    if( PARMETIS_MINOR_VERSION > 0 )
      return 0;
    if( PARMETIS_MAJOR_VERSION == 4 )
    if( PARMETIS_MINOR_VERSION == 0 )
    if( PARMETIS_SUBMINOR_VERSION >= 3 )
      return 0;
    return 1;
  }
  "
  )
  SET(CMAKE_REQUIRED_INCLUDES ${TPL_ParMETIS_INCLUDE_DIRS})
  SET(CMAKE_REQUIRED_LIBRARIES ${TPL_ParMETIS_LIBRARIES})
  SET(CMAKE_REQUIRED_FLAGS ${CMAKE_EXE_LINKER_FLAGS})
  CHECK_C_SOURCE_RUNS("${SOURCE}" ${VARNAME})
ENDFUNCTION()

The output above is from instrumenting the CHECK_C_SOURCE_RUNS so I can see why it is failing. You can obtain the same error by copy the source from CMakeFiles/CMakeError.log and compiling it.

I am a bit confused as to why other configure checks do not fail, or is it that Trilinos' CMake avoids checks that require executing a built binary?

@trilinos/zoltan2 @bartlettroscoe