1. PY-SCIPY crashes during installation

Spack has a ready-made recipe for installing the scipy module for Python, which is called py-scipy. During IMAS eco-system installation, while providing necessary python packages the one of which is the py-scipy, the installation of the py-scipy has crashed. The error message is ambiguous and is shown below:

pip._internal.exceptions.InstallationError: Could not build wheels for scipy, which is required to install pyproject.toml-based projects

1.1. How to reproduce issue

Log in to Marconi cluster
Load intel/pe-xe-2020–binary module
```
module load intel/pe-xe-2020--binary
```
Load intelmpi/2020--binary module
```
module load intelmpi/2020--binary
```

Clone spack repository

git clone https://github.com/spack/spack.git

To ensure the same environment set the packages.yaml file as below. This only adds packages marked as external:

the packages.yaml is located here:

vim ./spack/etc/spack/defaults/packages.yaml

And it should have following body:

# -------------------------------------------------------------------------
# This file controls default concretization preferences for Spack.
#
# Settings here are versioned with Spack and are intended to provide
# sensible defaults out of the box. Spack maintainers should edit this
# file to keep it current.
#
# Users can override these settings by editing the following files.
#
# Per-spack-instance settings (overrides defaults):
#   $SPACK_ROOT/etc/spack/packages.yaml
#
# Per-user settings (overrides default and site settings):
#   ~/.spack/packages.yaml
# -------------------------------------------------------------------------
packages:
  cmake:
   externals:
   - spec: cmake@3.23.0
     prefix: /marconi/home/userexternal/msmukows/spackdir/libs/cmake/3.23.0/gcc/7.3.0
  automake:
   externals:
   - spec: automake@1.16.5
     prefix: /marconi/home/userexternal/msmukows/spackdir/libs/automake/1.16.5/gcc/7.3.0
  autoconf:
   externals:
   - spec: autoconf@2.69
     prefix: /marconi/home/userexternal/msmukows/spackdir/libs/autoconf/2.69/gcc/7.3.0
  diffutils:
   externals:
   - spec: diffutils@3.8
     prefix: /marconi/home/userexternal/msmukows/spackdir/libs/diffutils/3.8/gcc/7.3.0
  perl:
   externals:
   - spec: perl@5.34.1
     prefix: /marconi/home/userexternal/msmukows/spackdir/libs/perl/5.34.1/gcc/7.3.0
  tar:
   externals:
   - spec: tar@1.34
     prefix: /marconi/home/userexternal/msmukows/spackdir/libs/tar/1.34/gcc/7.3.0
  gettext:
   externals:
   - spec: gettext@0.21
     prefix: /marconi/home/userexternal/msmukows/spackdir/libs/gettext/0.21/gcc/7.3.0
  libunwind:
   externals:
   - spec: libunwind@1.6.2
     prefix: /marconi/home/userexternal/msmukows/spackdir/libs/libunwind/1.6.2/gcc/7.3.0
  m4:
   externals:
   - spec: m4@1.4.19
     prefix: /marconi/home/userexternal/msmukows/spackdir/libs/m4/1.4.19/gcc/7.3.0
  bison:
   externals:
   - spec: bison@3.8.2
     prefix: /marconi/home/userexternal/msmukows/spackdir/libs/bison/3.8.2/gcc/7.3.0
  all:
    compiler: [gcc, intel, pgi, clang, xl, nag, fj, aocc]
    providers:
      awk: [gawk]
      blas: [openblas, amdblis]
      D: [ldc]
      daal: [intel-daal]
      elf: [elfutils]
      fftw-api: [fftw, amdfftw]
      flame: [libflame, amdlibflame]
      fuse: [libfuse]
      gl: [glx, osmesa]
      glu: [mesa-glu, openglu]
      golang: [go, gcc]
      go-external-or-gccgo-bootstrap: [go-bootstrap, gcc]
      iconv: [libiconv]
      ipp: [intel-ipp]
      java: [openjdk, jdk, ibm-java]
      jpeg: [libjpeg-turbo, libjpeg]
      lapack: [openblas, amdlibflame]
      libglx: [mesa+glx, mesa18+glx]
      libllvm: [llvm]
      libosmesa: [mesa+osmesa, mesa18+osmesa]
      lua-lang: [lua, lua-luajit-openresty, lua-luajit]
      luajit: [lua-luajit-openresty, lua-luajit]
      mariadb-client: [mariadb-c-client, mariadb]
      mkl: [intel-mkl]
      mpe: [mpe2]
      mpi: [openmpi, mpich]
      mysql-client: [mysql, mariadb-c-client]
      opencl: [pocl]
      onedal: [intel-oneapi-dal]
      pbs: [openpbs, torque]
      pil: [py-pillow]
      pkgconfig: [pkgconf, pkg-config]
      rpc: [libtirpc]
      scalapack: [netlib-scalapack, amdscalapack]
      sycl: [hipsycl]
      szip: [libaec, libszip]
      tbb: [intel-tbb]
      unwind: [libunwind]
      uuid: [util-linux-uuid, libuuid]
      xxd: [xxd-standalone, vim]
      yacc: [bison, byacc]
      ziglang: [zig]
    permissions:
      read: world
      write: user

Source SPACK to your environment:
```
source spack/share/spack/setup-env.sh
```

Install Python:

spack install python@3.9.10 +tkinter %intel@19.1.3.304 cflags="-fPIC" cxxflags="-fPIC" cppflags="-fPIC"

Install py-matplotlib:

spack install --fail-fast py-matplotlib backend=tkagg %intel@19.1.3.304 cflags="-fPIC" cxxflags="-fPIC" cppflags="-fPIC"

Install py-cython:

spack install --fail-fast py-cython %intel@19.1.3.304 cflags="-fPIC" cxxflags="-fPIC" cppflags="-fPIC"

Install py-pyyaml:

spack install --fail-fast py-pyyaml %intel@19.1.3.304 cflags="-fPIC" cxxflags="-fPIC" cppflags="-fPIC"

Install py-scipy:

spack install --fail-fast py-scipy %intel@19.1.3.304 cflags="-fPIC" cxxflags="-fPIC" cppflags="-fPIC"

During this installation you should get an error:

pip._internal.exceptions.InstallationError: Could not build wheels for scipy, which is required to install pyproject.toml-based projects

1.2. Solution

After many attempts and tests, a solution was found. The py-scipy package must be the first Python package to be installed before any other.

Log in to Marconi cluster
Load intel/pe-xe-2020–binary module
```
module load intel/pe-xe-2020--binary
```
Load intelmpi/2020--binary module
```
module load intelmpi/2020--binary
```