Skip to content
GitLab

Install dependencies with Spack

Open Elliott Slaughter requested to merge eds/spack into development

This MR changes our build infrastructure to use Spack:

An overview of the configuration:

  • setup/spack.yaml: Contains the system independent configuration. This is mostly a shim around the py-spinifel Spack package. Do not add new dependencies here.
  • setup/spack-repo/packages/py-spinifel/package.py: Contains the guts of our Spinifel package. Note that this is just a "bundle" package which means it contains dependencies only and no real build step. This is where new dependencies should be added.
  • setup/machines/${MACHINE}.yaml: Contains the system dependent configuration. Should be limited to compilers, packages and modules sections only. (E.g., this is where we tell it what compiler and MPI to use, where we enable CUDA/ROCm, etc.)
  • setup/machines/${MACHINE}.lock: The concretized environment for a given machine. Should contain the exact versions of all dependencies (including transitive dependencies). In theory, if we can build it once, we can reproduce this any time.
  • setup/spack: The repository of Spack itself. Currently pinned to a recent develop branch commit because we need very recent changes. (Edit: moved to a fork temporarily because we need changes not yet merged upstream.)

Some other notable features of this change:

  • I ripped out a lot of machine-specific configuration. To the extent possible, I want everything to be done with Spack now. We can move even more into Spack later, but this is a start.
  • In particular, we do NOT use the system-specific FFTW. This has been moved into Spack. (This means giving up on the Cray FFT library. I think this is fine in practice because we will never run on a Cray machine that does not have GPUs.)
  • I removed a bunch of obsolete machines (e.g., Cori and Cori GPU) that do not exist anymore.
  • In order to make (re)builds fast, I assume that a build cache has been established on every machine. This is just a directory in a shared location (like a project home directory) that is group-writable and special guid bit set.

The build has been confirmed to work on Perlmutter, Crusher and Frontier. CI is passing on Crusher. There are issues on Summit/Ascent, but that machine is going away soon so I don't plan to fix them.

Edited by Elliott Slaughter

Merge request reports