Tpetra: gemm C wrappers mangle to same names as Fortran when built with XL compiler
Created by: djfitzgerald
@trilinos/Tpetra
Expectations
Attempt to build trilinos
with tpetra
support using the IBM XL compiler succeeds.
Current Behavior
Using Spack, I attempted to build and install trilinos
with tpetra
support using the IBM XL compiler. The specific Spack command was: spack install --test=root trilinos%xl_r ~pnetcdf ^netlib-lapack+external-blas ^netcdf%gcc ^m4%gcc
, which will attempt to build and install trilinos
with the parallel IBM XL compiler, without pnetcdf
support, with netlib-lapack
support using an external BLAS provider (in my case, IBM ESSL), with netcdf
and m4
dependencies built with gcc, and then perform any cmake
automated testing for the trilinos
package.
Spack produces the following error output:
==> Building trilinos [CMakePackage]
==> Executing phase: 'cmake'
==> Executing phase: 'build'
==> Error: ProcessError: Command exited with status 2:
'make'
6 errors found in build log:
[ ... ]
1864 [ 35%] Building CXX object packages/tpetra/core/src/CMakeFiles/tpetra.dir/Tpetra_Details_iallreduce.cpp.o
1865 cd /tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/tpetra/core/src && /projects/opt/ppc64le/openmpi/2.1.0-gcc_7.1.0/bin/mpic++ -Dtpetra_EXPORTS -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/tpetra/core/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/tpetra/core/src/kokkos_refactor -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/tpetra/core/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/tpetra/tsqr/src -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/tpetra/tsqr/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/tpetra/classic/LinAlg -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/tpetra/classic/NodeAPI -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/tpetra/classic/NodeAPI -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/tpetra/classic/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/tpetra/classic/src -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/teuchos/kokkoscomm/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/teuchos/kokkoscomm/src -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/teuchos/kokkoscompat/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/teuchos/kokkoscompat/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/teuchos/parameterlist/src -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/teuchos/core/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/teuchos/core/src -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/kokkos/core/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/kokkos/core/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/teuchos/comm/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/teuchos/remainder/src -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/teuchos/remainder/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/teuchos/numerics/src -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/epetra/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/epetra/src -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/kokkos-kernels/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/kokkos-kernels/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/kokkos-kernels/src/impl -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/kokkos-kernels/src/stage/graph -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/kokkos-kernels/src/stage/graph/impl -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/kokkos-kernels/src/stage/graph/utils -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/kokkos/algorithms/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/kokkos/algorithms/src -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/kokkos/containers/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/kokkos/containers/src -std=c++11 -O2 -g -DNDEBUG -fPIC -o CMakeFiles/tpetra.dir/Tpetra_Details_iallreduce.cpp.o -c /projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/tpetra/core/src/Tpetra_Details_iallreduce.cpp
1866 [ 35%] Building CXX object packages/tpetra/core/src/CMakeFiles/tpetra.dir/Tpetra_Details_isInterComm.cpp.o
1867 cd /tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/tpetra/core/src && /projects/opt/ppc64le/openmpi/2.1.0-gcc_7.1.0/bin/mpic++ -Dtpetra_EXPORTS -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/tpetra/core/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/tpetra/core/src/kokkos_refactor -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/tpetra/core/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/tpetra/tsqr/src -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/tpetra/tsqr/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/tpetra/classic/LinAlg -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/tpetra/classic/NodeAPI -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/tpetra/classic/NodeAPI -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/tpetra/classic/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/tpetra/classic/src -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/teuchos/kokkoscomm/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/teuchos/kokkoscomm/src -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/teuchos/kokkoscompat/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/teuchos/kokkoscompat/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/teuchos/parameterlist/src -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/teuchos/core/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/teuchos/core/src -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/kokkos/core/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/kokkos/core/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/teuchos/comm/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/teuchos/remainder/src -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/teuchos/remainder/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/teuchos/numerics/src -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/epetra/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/epetra/src -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/kokkos-kernels/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/kokkos-kernels/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/kokkos-kernels/src/impl -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/kokkos-kernels/src/stage/graph -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/kokkos-kernels/src/stage/graph/impl -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/kokkos-kernels/src/stage/graph/utils -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/kokkos/algorithms/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/kokkos/algorithms/src -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/kokkos/containers/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/kokkos/containers/src -std=c++11 -O2 -g -DNDEBUG -fPIC -o CMakeFiles/tpetra.dir/Tpetra_Details_isInterComm.cpp.o -c /projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/tpetra/core/src/Tpetra_Details_isInterComm.cpp
1868 [ 35%] Building CXX object packages/tpetra/core/src/CMakeFiles/tpetra.dir/Tpetra_Details_libGemm.cpp.o
1869 cd /tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/tpetra/core/src && /projects/opt/ppc64le/openmpi/2.1.0-gcc_7.1.0/bin/mpic++ -Dtpetra_EXPORTS -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/tpetra/core/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/tpetra/core/src/kokkos_refactor -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/tpetra/core/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/tpetra/tsqr/src -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/tpetra/tsqr/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/tpetra/classic/LinAlg -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/tpetra/classic/NodeAPI -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/tpetra/classic/NodeAPI -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/tpetra/classic/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/tpetra/classic/src -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/teuchos/kokkoscomm/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/teuchos/kokkoscomm/src -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/teuchos/kokkoscompat/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/teuchos/kokkoscompat/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/teuchos/parameterlist/src -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/teuchos/core/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/teuchos/core/src -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/kokkos/core/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/kokkos/core/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/teuchos/comm/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/teuchos/remainder/src -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/teuchos/remainder/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/teuchos/numerics/src -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/epetra/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/epetra/src -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/kokkos-kernels/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/kokkos-kernels/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/kokkos-kernels/src/impl -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/kokkos-kernels/src/stage/graph -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/kokkos-kernels/src/stage/graph/impl -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/kokkos-kernels/src/stage/graph/utils -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/kokkos/algorithms/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/kokkos/algorithms/src -I/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/kokkos/containers/src -I/projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/kokkos/containers/src -std=c++11 -O2 -g -DNDEBUG -fPIC -o CMakeFiles/tpetra.dir/Tpetra_Details_libGemm.cpp.o -c /projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/tpetra/core/src/Tpetra_Details_libGemm.cpp
>> 1870 /projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/tpetra/core/src/Tpetra_Details_libGemm.cpp:148:3: error: no matching function for call to 'dgemm'
1871 TPETRACORE_DGEMM (&transA, &transB, &m, &n, &k, &alpha, A, &lda, B, &ldb, &beta, C, &ldc);
1872 ^~~~~~~~~~~~~~~~
1873 /projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/tpetra/core/src/Tpetra_Details_libGemm.cpp:70:53: note: expanded from macro 'TPETRACORE_DGEMM'
1874 #define TPETRACORE_DGEMM TPETRACORE_F77_BLAS_MANGLE(dgemm,DGEMM)
1875 ^~~~~
1876 /tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/tpetra/core/src/TpetraCore_config.h:171:49: note: expanded from macro 'TPETRACORE_F77_BLAS_MANGLE'
1877 # define TPETRACORE_F77_BLAS_MANGLE(name,NAME) name
1878 ^~~~
1879 /projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/tpetra/core/src/Tpetra_Details_libGemm.cpp:134:1: note: candidate function not viable: no known conversion from 'const char *' to 'const char' for 1st argument; remove &
1880 dgemm (const char transA,
1881 ^
>> 1882 /projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/tpetra/core/src/Tpetra_Details_libGemm.cpp:166:3: error: no matching function for call to 'sgemm'
1883 TPETRACORE_SGEMM (&transA, &transB, &m, &n, &k, &alpha, A, &lda, B, &ldb, &beta, C, &ldc);
1884 ^~~~~~~~~~~~~~~~
1885 /projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/tpetra/core/src/Tpetra_Details_libGemm.cpp:80:53: note: expanded from macro 'TPETRACORE_SGEMM'
1886 #define TPETRACORE_SGEMM TPETRACORE_F77_BLAS_MANGLE(sgemm,SGEMM)
1887 ^~~~~
1888 /tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build/packages/tpetra/core/src/TpetraCore_config.h:171:49: note: expanded from macro 'TPETRACORE_F77_BLAS_MANGLE'
1889 # define TPETRACORE_F77_BLAS_MANGLE(name,NAME) name
1890 ^~~~
1891 /projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/tpetra/core/src/Tpetra_Details_libGemm.cpp:152:1: note: candidate function not viable: no known conversion from 'const char *' to 'const char' for 1st argument; remove &
1892 sgemm (const char transA,
1893 ^
1894 2 errors generated.
>> 1895 Error while processing /projects/draco/vendors/spack.power8-0.11.1/var/spack/stage/trilinos-12.12.1-rdm5vlkvnusztnxqsjmpd4tiaw4k7uxr/Trilinos-trilinos-release-12-12-1/packages/tpetra/core/src/Tpetra_Details_libGemm.cpp.
>> 1896 make[2]: *** [packages/tpetra/core/src/CMakeFiles/tpetra.dir/Tpetra_Details_libGemm.cpp.o] Error 1
1897 make[2]: Leaving directory `/ram/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build'
>> 1898 make[1]: *** [packages/tpetra/core/src/CMakeFiles/tpetra.dir/all] Error 2
1899 make[1]: Leaving directory `/ram/tmp/dantopa/spack-stage/spack-stage-Rv6nqz/Trilinos-trilinos-release-12-12-1/spack-build'
>> 1900 make: *** [all] Error 2
Motivation and Context
This prevented my team from being able to compile and install the Trilinos package with Tpetra enabled on an IBM Power9 system using the IBM XL compiler suite. Being able to do so was a hard requirement provided to us by our customer. My team had to debug the problem and develop a patch which may or may not be the optimal solution for this defect.
Definition of Done
Correct mangling error in C/Fortran interface code (see "Possible Solution" section below). Spack must be able to build and install Trilinos with Tpetra support using the IBM XL compiler, and must be able to run Trilinos' automated cmake
verification tests.
Possible Solution
The cause of this problem is that gfortran
appends '_' to external subroutine names in *.o
files but xlf
does not. dgemm
is apparently a Fortran subroutine, and Tpetra_Details_libGemm.cpp
was developed as a C wrapper to the *gemm interfaces. Because Fortran is pass-by-reference and C/C++ pass-by-value, the wrappers in Tpetra_Details_libGemm.cpp
allow C/C++ callers to call the *gemm functions with values, and invoke the actual Fortran *gemm functions with pointers to those values . The wrapper function uses the TPETRACORE_F77_BLAS_MANGLE
macro to determine the C-mangled name of the Fortran function.
TPETRACORE_F77_BLAS_MANGLE
is built during configure time when cmake
presumably figures out how the Fortran compiler builds external function names in *.o
files. In the gfortran
case, TPETRACORE_DGEMM TPETRACORE_F77_BLAS_MANGLE(dgemm,DGEMM)
becomes dgemm_
, and all goes well. However, in the xlf
case, this resolves to dgemm
without the underbar, which happens to be the same name as the dgemm
wrapper function defined in the Tpetra_Details_libGemm.cpp
source file. Also, since the TPETRACORE_DGEMM
function prototype is declared in the scope of extern "C"
, mangling rules don't apply for the local definition either. So now you have a prototype that takes pointer arguments and a function definition that takes value arguments, which causes the error that I observed.
My fix was to redefine the C/C++ wrapper function names as *gemm_wrap
, eliminating the possibility that the wrapper names could collide with the actual Fortran function names. This seems to get us past this problem, although I admit thorough testing has not been done as I am not intimately familiar with Trilinos and its' use cases.
Steps to Reproduce
- Install IBM XL 16.1 on a test system per product installation instructions.
- Extract Spack into your home directory on the test system by extracting the *.zip file attached to this issue. Newer versions will include my patch for this problem, preventing recreation.
- From your home directory, run
. ~/spack/share/spack/setup-env.sh ; export PATH=$PATH:$HOME/spack
- Issue
spack compilers
and verify that Spack has detected the IBM XL compilers and gcc > 4.9.0. If your gcc is less than 4.9.0, do the following: 4a.spack install gcc@5.1.0
4b. When Spack has installed gcc 5.1.0, it will display a line of output indicating where it was installed to. Copy that path. 4c.spack compiler add GCCPATH
where GCCPATH is the path you copied in step 4b. 4d. Issuespack compilers
and verify that Spack has detected gcc 5.1.0. - Attempt to install Trilinos through Spack:
spack install trilinos%xl_r ~pnetcdf ^netlib-lapack+external-blas ^netcdf%gcc ^m4%gcc
Your Environment
- Built using Trilinos 12.12.1 from git (SHA ecd4606fa332212433c98bf950a69cc7)
- Operating system was RHEL 7.4 on an IBM Power9 system.
- XL compiler was xl_r from IBM XL 16.1 compiler suite, gcc compiler was 5.1.0.
Related Issues
This was originally opened as Spack issue 7247.
Additional Information
My patch to fix this problem is attached, as is a version of Spack without my fix. xlf_tpetra.patch.txt spack-develop.zip