1. PY-SCIPY crashes during installation
Spack has a ready-made recipe for installing the scipy
module for Python, which is called py-scipy
. During IMAS eco-system installation, while providing necessary python packages the one of which is the py-scipy,
the installation of the py-scipy
has crashed. The error message is ambiguous and is shown below:
pip._internal.exceptions.InstallationError: Could not build wheels for scipy, which is required to install pyproject.toml-based projects
1.1. How to reproduce issue
- Log in to Marconi cluster
Load
intel/pe-xe-2020–binary
modulemodule load intel/pe-xe-2020--binary
Load
intelmpi/2020--binary
modulemodule load intelmpi/2020--binary
Clone spack repository
git clone https://github.com/spack/spack.git
To ensure the same environment set the
packages.yaml
file as below. This only adds packages marked as external:Source SPACK to your environment:
source spack/share/spack/setup-env.sh
Install Python:
spack install python@3.9.10 +tkinter %intel@19.1.3.304 cflags="-fPIC" cxxflags="-fPIC" cppflags="-fPIC"
Install py-matplotlib:
spack install --fail-fast py-matplotlib backend=tkagg %intel@19.1.3.304 cflags="-fPIC" cxxflags="-fPIC" cppflags="-fPIC"
Install py-cython:
spack install --fail-fast py-cython %intel@19.1.3.304 cflags="-fPIC" cxxflags="-fPIC" cppflags="-fPIC"
Install py-pyyaml:
spack install --fail-fast py-pyyaml %intel@19.1.3.304 cflags="-fPIC" cxxflags="-fPIC" cppflags="-fPIC"
Install py-scipy:
spack install --fail-fast py-scipy %intel@19.1.3.304 cflags="-fPIC" cxxflags="-fPIC" cppflags="-fPIC"
During this installation you should get an error:
pip._internal.exceptions.InstallationError: Could not build wheels for scipy, which is required to install pyproject.toml-based projects
1.2. Solution
After many attempts and tests, a solution was found. The py-scipy
package must be the first Python package to be installed before any other.
- Log in to Marconi cluster
Load
intel/pe-xe-2020–binary
modulemodule load intel/pe-xe-2020--binary
Load
intelmpi/2020--binary
modulemodule load intelmpi/2020--binary
Clone spack repository
git clone https://github.com/spack/spack.git
To ensure the same environment set the
packages.yaml
file as below. This only adds packages marked as external:Source SPACK to your environment:
source spack/share/spack/setup-env.sh
Install Python:
spack install python@3.9.10 +tkinter %intel@19.1.3.304 cflags="-fPIC" cxxflags="-fPIC" cppflags="-fPIC"
Install py-scipy:
spack install --fail-fast py-scipy %intel@19.1.3.304 cflags="-fPIC" cxxflags="-fPIC" cppflags="-fPIC"
Install py-matplotlib:
spack install --fail-fast py-matplotlib backend=tkagg %intel@19.1.3.304 cflags="-fPIC" cxxflags="-fPIC" cppflags="-fPIC"
Install py-cython:
spack install --fail-fast py-cython %intel@19.1.3.304 cflags="-fPIC" cxxflags="-fPIC" cppflags="-fPIC"
Install py-pyyaml:
spack install --fail-fast py-pyyaml %intel@19.1.3.304 cflags="-fPIC" cxxflags="-fPIC" cppflags="-fPIC"