Teuchos: Address issus with calling Fortran LAPACK rountines from C++ on Power8 with GCC/gfortran

Created by: mhoemmen

Next Action Status:

Looks like it was a problem with BLAS and LAPACK after all (see https://github.com/trilinos/Trilinos/issues/2454#issuecomment-386271621).

Blocks: #1191 (closed) Duplicates: #347 (closed) Analogous to: #1210 CC: @trilinos/teuchos @trilinos/framework

Description:

NOTE: The below original description does not pin-point the actual problem that was causing the failures on 'ride'. The real problem is what appears to be a mixed language calling defect summarized below.

Original Description:

BLAS and LAPACK libraries offer a Fortran 77 interface, but we want to call them from C++ code. This matters both in terms of Fortran function name mangling, and in terms of how to pass arguments into Fortran functions (the "application binary interface" or ABI). Most arguments just go into Fortran from the C or C++ world by pointer. However, some Fortran compilers have a different ABI for passing strings (CHARACTER(*)). Some compilers just take char*, but others may take char*, unsigned int or even char*, unsigned int* (or not unsigned). Fortran programmers don't see this; only programmers working in other languages (like C or C++), who need to know the Fortran ABI, may see this.

Teuchos' BLAS and LAPACK wrappers have two macros that govern the Fortran string ABI: CHAR_MACRO(c) and Teuchos_fcd.

CHAR_MACRO(c), defined redundantly in teuchos/numerics/src/Teuchos_LAPACK.cpp and Teuchos_BLAS.cpp, takes a const char c and does the right thing with it to pass it into a Fortran function. Teuchos offers two definitions of CHAR_MACRO:

#if defined (INTEL_CXML)
#define CHAR_MACRO(char_var) &char_var, one
#else
#define CHAR_MACRO(char_var) &char_var
#endif

namespace {
#if defined (INTEL_CXML)
        unsigned int one=1;
#endif
// ...
}

Per discussion in #1191 (closed), it looks like we need another definition for OpenBLAS on POWER, that passes in the constant one by address rather than by value.

Teuchos_fcd, defined redundantly in teuchos/numerics/src/Teuchos_LAPACK_wrappers.hpp and Teuchos_BLAS_wrappers.hpp, has the following definition:

#if defined(INTEL_CXML)
#  define PREFIX __stdcall
#  define Teuchos_fcd const char *, unsigned int
#elif defined(INTEL_MKL)
#  define PREFIX
#  define Teuchos_fcd const char *
#else /* Not CRAY_T3X or INTEL_CXML or INTEL_MKL */
#  define PREFIX
#  define Teuchos_fcd const char *
#endif

In the comments, CRAY_T3X refers to the Cray T3D and T3E, two mid-'90s distributed-memory computer architectures. This should give you an idea of the age of this code. In any case, we would need to add a new definition of Teuchos_fcd:

#  define Teuchos_fcd const char*, const int*

(See comments below for why it's an int and not unsigned int. It would be better to use int32_t, but Teuchos cannot assume C++11 / C99 types.)