Table of Contents |
---|
PY-SCIPY crashes during installation
Spack has a ready-made recipe for installing the scipy
module for Python, which is called py-scipy
. During IMAS eco-system installation, while providing necessary python packages the one of which is the py-scipy,
the installation of the py-scipy
has crashed. The error message is ambiguous and is shown below:
Code Block |
---|
pip._internal.exceptions.InstallationError: Could not build wheels for scipy, which is required to install pyproject.toml-based projects |
How to reproduce issue
- Log in to Marconi cluster
Load
intel/pe-xe-2020–binary
moduleCode Block module load intel/pe-xe-2020--binary
Load
intelmpi/2020--binary
moduleCode Block module load intelmpi/2020--binary
Clone spack repository
Code Block git clone https://github.com/spack/spack.git
To ensure the same environment set the
packages.yaml
file as below. This only adds packages marked as external:Expand the
packages.yaml
is located here:Code Block vim ./spack/etc/spack/defaults/packages.yaml
And it should have following body:
Code Block # ------------------------------------------------------------------------- # This file controls default concretization preferences for Spack. # # Settings here are versioned with Spack and are intended to provide # sensible defaults out of the box. Spack maintainers should edit this # file to keep it current. # # Users can override these settings by editing the following files. # # Per-spack-instance settings (overrides defaults): # $SPACK_ROOT/etc/spack/packages.yaml # # Per-user settings (overrides default and site settings): # ~/.spack/packages.yaml # ------------------------------------------------------------------------- packages: cmake: externals: - spec: cmake@3.23.0 prefix: /marconi/home/userexternal/msmukows/spackdir/libs/cmake/3.23.0/gcc/7.3.0 automake: externals: - spec: automake@1.16.5 prefix: /marconi/home/userexternal/msmukows/spackdir/libs/automake/1.16.5/gcc/7.3.0 autoconf: externals: - spec: autoconf@2.69 prefix: /marconi/home/userexternal/msmukows/spackdir/libs/autoconf/2.69/gcc/7.3.0 diffutils: externals: - spec: diffutils@3.8 prefix: /marconi/home/userexternal/msmukows/spackdir/libs/diffutils/3.8/gcc/7.3.0 perl: externals: - spec: perl@5.34.1 prefix: /marconi/home/userexternal/msmukows/spackdir/libs/perl/5.34.1/gcc/7.3.0 tar: externals: - spec: tar@1.34 prefix: /marconi/home/userexternal/msmukows/spackdir/libs/tar/1.34/gcc/7.3.0 gettext: externals: - spec: gettext@0.21 prefix: /marconi/home/userexternal/msmukows/spackdir/libs/gettext/0.21/gcc/7.3.0 libunwind: externals: - spec: libunwind@1.6.2 prefix: /marconi/home/userexternal/msmukows/spackdir/libs/libunwind/1.6.2/gcc/7.3.0 m4: externals: - spec: m4@1.4.19 prefix: /marconi/home/userexternal/msmukows/spackdir/libs/m4/1.4.19/gcc/7.3.0 bison: externals: - spec: bison@3.8.2 prefix: /marconi/home/userexternal/msmukows/spackdir/libs/bison/3.8.2/gcc/7.3.0 all: compiler: [gcc, intel, pgi, clang, xl, nag, fj, aocc] providers: awk: [gawk] blas: [openblas, amdblis] D: [ldc] daal: [intel-daal] elf: [elfutils] fftw-api: [fftw, amdfftw] flame: [libflame, amdlibflame] fuse: [libfuse] gl: [glx, osmesa] glu: [mesa-glu, openglu] golang: [go, gcc] go-external-or-gccgo-bootstrap: [go-bootstrap, gcc] iconv: [libiconv] ipp: [intel-ipp] java: [openjdk, jdk, ibm-java] jpeg: [libjpeg-turbo, libjpeg] lapack: [openblas, amdlibflame] libglx: [mesa+glx, mesa18+glx] libllvm: [llvm] libosmesa: [mesa+osmesa, mesa18+osmesa] lua-lang: [lua, lua-luajit-openresty, lua-luajit] luajit: [lua-luajit-openresty, lua-luajit] mariadb-client: [mariadb-c-client, mariadb] mkl: [intel-mkl] mpe: [mpe2] mpi: [openmpi, mpich] mysql-client: [mysql, mariadb-c-client] opencl: [pocl] onedal: [intel-oneapi-dal] pbs: [openpbs, torque] pil: [py-pillow] pkgconfig: [pkgconf, pkg-config] rpc: [libtirpc] scalapack: [netlib-scalapack, amdscalapack] sycl: [hipsycl] szip: [libaec, libszip] tbb: [intel-tbb] unwind: [libunwind] uuid: [util-linux-uuid, libuuid] xxd: [xxd-standalone, vim] yacc: [bison, byacc] ziglang: [zig] permissions: read: world write: user
Source SPACK to your environment:
Code Block source spack/share/spack/setup-env.sh
Install Python:
Code Block spack install python@3.9.10 +tkinter %intel@19.1.3.304 cflags="-fPIC" cxxflags="-fPIC" cppflags="-fPIC"
Install py-matplotlib:
Code Block spack install --fail-fast py-matplotlib backend=tkagg %intel@19.1.3.304 cflags="-fPIC" cxxflags="-fPIC" cppflags="-fPIC"
Install py-cython:
Code Block spack install --fail-fast py-cython %intel@19.1.3.304 cflags="-fPIC" cxxflags="-fPIC" cppflags="-fPIC"
Install py-pyyaml:
Code Block spack install --fail-fast py-pyyaml %intel@19.1.3.304 cflags="-fPIC" cxxflags="-fPIC" cppflags="-fPIC"
Install py-scipy:
Code Block spack install --fail-fast py-scipy %intel@19.1.3.304 cflags="-fPIC" cxxflags="-fPIC" cppflags="-fPIC"
During this installation you should get an error:
Code Block pip._internal.exceptions.InstallationError: Could not build wheels for scipy, which is required to install pyproject.toml-based projects
Solution
After many attempts and tests, a solution was found. The py-scipy
package must be the first Python package to be installed before any other.
- Log in to Marconi cluster
Load
intel/pe-xe-2020–binary
moduleCode Block module load intel/pe-xe-2020--binary
Load
intelmpi/2020--binary
moduleCode Block module load intelmpi/2020--binary
Clone spack repository
Code Block git clone https://github.com/spack/spack.git
To ensure the same environment set the
packages.yaml
file as below. This only adds packages marked as external:Expand the
packages.yaml
is located here:Code Block vim ./spack/etc/spack/defaults/packages.yaml
And it should have following body:
Code Block # ------------------------------------------------------------------------- # This file controls default concretization preferences for Spack. # # Settings here are versioned with Spack and are intended to provide # sensible defaults out of the box. Spack maintainers should edit this # file to keep it current. # # Users can override these settings by editing the following files. # # Per-spack-instance settings (overrides defaults): # $SPACK_ROOT/etc/spack/packages.yaml # # Per-user settings (overrides default and site settings): # ~/.spack/packages.yaml # ------------------------------------------------------------------------- packages: cmake: externals: - spec: cmake@3.23.0 prefix: /marconi/home/userexternal/msmukows/spackdir/libs/cmake/3.23.0/gcc/7.3.0 automake: externals: - spec: automake@1.16.5 prefix: /marconi/home/userexternal/msmukows/spackdir/libs/automake/1.16.5/gcc/7.3.0 autoconf: externals: - spec: autoconf@2.69 prefix: /marconi/home/userexternal/msmukows/spackdir/libs/autoconf/2.69/gcc/7.3.0 diffutils: externals: - spec: diffutils@3.8 prefix: /marconi/home/userexternal/msmukows/spackdir/libs/diffutils/3.8/gcc/7.3.0 perl: externals: - spec: perl@5.34.1 prefix: /marconi/home/userexternal/msmukows/spackdir/libs/perl/5.34.1/gcc/7.3.0 tar: externals: - spec: tar@1.34 prefix: /marconi/home/userexternal/msmukows/spackdir/libs/tar/1.34/gcc/7.3.0 gettext: externals: - spec: gettext@0.21 prefix: /marconi/home/userexternal/msmukows/spackdir/libs/gettext/0.21/gcc/7.3.0 libunwind: externals: - spec: libunwind@1.6.2 prefix: /marconi/home/userexternal/msmukows/spackdir/libs/libunwind/1.6.2/gcc/7.3.0 m4: externals: - spec: m4@1.4.19 prefix: /marconi/home/userexternal/msmukows/spackdir/libs/m4/1.4.19/gcc/7.3.0 bison: externals: - spec: bison@3.8.2 prefix: /marconi/home/userexternal/msmukows/spackdir/libs/bison/3.8.2/gcc/7.3.0 all: compiler: [gcc, intel, pgi, clang, xl, nag, fj, aocc] providers: awk: [gawk] blas: [openblas, amdblis] D: [ldc] daal: [intel-daal] elf: [elfutils] fftw-api: [fftw, amdfftw] flame: [libflame, amdlibflame] fuse: [libfuse] gl: [glx, osmesa] glu: [mesa-glu, openglu] golang: [go, gcc] go-external-or-gccgo-bootstrap: [go-bootstrap, gcc] iconv: [libiconv] ipp: [intel-ipp] java: [openjdk, jdk, ibm-java] jpeg: [libjpeg-turbo, libjpeg] lapack: [openblas, amdlibflame] libglx: [mesa+glx, mesa18+glx] libllvm: [llvm] libosmesa: [mesa+osmesa, mesa18+osmesa] lua-lang: [lua, lua-luajit-openresty, lua-luajit] luajit: [lua-luajit-openresty, lua-luajit] mariadb-client: [mariadb-c-client, mariadb] mkl: [intel-mkl] mpe: [mpe2] mpi: [openmpi, mpich] mysql-client: [mysql, mariadb-c-client] opencl: [pocl] onedal: [intel-oneapi-dal] pbs: [openpbs, torque] pil: [py-pillow] pkgconfig: [pkgconf, pkg-config] rpc: [libtirpc] scalapack: [netlib-scalapack, amdscalapack] sycl: [hipsycl] szip: [libaec, libszip] tbb: [intel-tbb] unwind: [libunwind] uuid: [util-linux-uuid, libuuid] xxd: [xxd-standalone, vim] yacc: [bison, byacc] ziglang: [zig] permissions: read: world write: user
Source SPACK to your environment:
Code Block source spack/share/spack/setup-env.sh
Install Python:
Code Block spack install python@3.9.10 +tkinter %intel@19.1.3.304 cflags="-fPIC" cxxflags="-fPIC" cppflags="-fPIC"
Install py-scipy:
Code Block spack install --fail-fast py-scipy %intel@19.1.3.304 cflags="-fPIC" cxxflags="-fPIC" cppflags="-fPIC"
Install py-matplotlib:
Code Block spack install --fail-fast py-matplotlib backend=tkagg %intel@19.1.3.304 cflags="-fPIC" cxxflags="-fPIC" cppflags="-fPIC"
Install py-cython:
Code Block spack install --fail-fast py-cython %intel@19.1.3.304 cflags="-fPIC" cxxflags="-fPIC" cppflags="-fPIC"
Install py-pyyaml:
Code Block spack install --fail-fast py-pyyaml %intel@19.1.3.304 cflags="-fPIC" cxxflags="-fPIC" cppflags="-fPIC"
BLITZ and missing -pthread
flag
After installing all dependencies, the very next step is to install IMAS itself. During this installation the cpp interface component is complaining about blitz not being compiled in thread-safe
manner and suggests that maybe the -pthread
flag was missing during Blitz compilation/installation. The bad news is that the whole eco-system was compiled using at least those two flags: '-fPIC -pthread'
, therefore the error is confusing. Below you can find an error message from reproduction attempt:
Code Block |
---|
make -C ./src/3.36.0/ual/4.10.1/examples/cpp all || exit 1
make[2]: Entering directory `/marconi_work/FUSIO_HLST/msmukows/IMAS_INSTALLER/intel_installer/src/3.36.0/ual/4.10.1/examples/cpp'
icpc `pkg-config blitz --cflags` -I/marconi_work/FUSIO_HLST/msmukows/IMAS_INSTALLER/intel_installer/src/3.36.0/ual/4.10.1/cppinterface/src -I/marconi_work/FUSIO_HLST/msmukows/IMAS_INSTALLER/intel_installer/src/3.36.0/ual/4.10.1/lowlevel -g -O0 -fPIC -shared-intel -c test_pulse_create.cpp
In file included from /marconi_work/FUSIO_HLST/switmdir/libs-intel/blitz/1.0.1/intel/19.1.3.304/hftjglypfr4ovoej6dkxwuedupjqez3m/include/blitz/array-impl.h(46),
from /marconi_work/FUSIO_HLST/switmdir/libs-intel/blitz/1.0.1/intel/19.1.3.304/hftjglypfr4ovoej6dkxwuedupjqez3m/include/blitz/array.h(37),
from /marconi_work/FUSIO_HLST/msmukows/IMAS_INSTALLER/intel_installer/src/3.36.0/ual/4.10.1/cppinterface/src/UALDef.h(30),
from /marconi_work/FUSIO_HLST/msmukows/IMAS_INSTALLER/intel_installer/src/3.36.0/ual/4.10.1/cppinterface/src/UALClasses.h(7),
from test_pulse_create.cpp(1):
/marconi_work/FUSIO_HLST/switmdir/libs-intel/blitz/1.0.1/intel/19.1.3.304/hftjglypfr4ovoej6dkxwuedupjqez3m/include/blitz/blitz.h(152): error: #error directive: Blitz is configured with --enable-threadsafe, but no compiler thread support is found. Did you forget, e.g., "--pthread"?
#error Blitz is configured with --enable-threadsafe, but no compiler thread support is found. Did you forget, e.g., "--pthread"?
^
compilation aborted for test_pulse_create.cpp (code 2)
make[2]: *** [test_pulse_create.o] Error 2
make[2]: Leaving directory `/marconi_work/FUSIO_HLST/msmukows/IMAS_INSTALLER/intel_installer/src/3.36.0/ual/4.10.1/examples/cpp'
make[1]: *** [examples/cpp] Error 1
make[1]: Leaving directory `/marconi_work/FUSIO_HLST/msmukows/IMAS_INSTALLER/intel_installer'
make: *** [all] Error 2 |
As you can see for some reason IMAS is complaining on Blitz being compiled with --enable-threadsafe
flag but at the same time no thread support for compiler is found:
Code Block |
---|
/marconi_work/FUSIO_HLST/switmdir/libs-intel/blitz/1.0.1/intel/19.1.3.304/hftjglypfr4ovoej6dkxwuedupjqez3m/include/blitz/blitz.h(152): error: #error directive: Blitz is configured with --enable-threadsafe, but no compiler thread support is found. Did you forget, e.g., "--pthread"?
#error Blitz is configured with --enable-threadsafe, but no compiler thread support is found. Did you forget, e.g., "--pthread"? |
How to reproduce issue
To avoid installing whole IMAS-Spack eco-system from scratch a minimal working example was prepared. This just takes the part of the IMAS code and tries to compile it the very same way as the IMAS installer does it.
- Log in to Marconi cluster
Prepare
$MODULEPATH
thus the imasenv module can be found and loaded:Code Block module use /marconi/home/userexternal/msmukows/public/modules
Load
imasenv/3.36.0/intel/2020-test
moduleCode Block module load imasenv/3.36.0/intel/2020-test
Navigate to the directory with example code:
Code Block cd /marconi_work/FUSIO_HLST/testdir/tests/blitz_pthread_issue_intel_2020
- There should be a file named:
"test_pulse_create.cpp":
Code Block . └── test_pulse_create.cpp
Now execute compilation command:
Code Block icpc `pkg-config blitz --cflags` -I/marconi_work/FUSIO_HLST/msmukows/IMAS_INSTALLER/intel_installer/src/3.36.0/ual/4.10.1/cppinterface/src -I/marconi_work/FUSIO_HLST/msmukows/IMAS_INSTALLER/intel_installer/src/3.36.0/ual/4.10.1/lowlevel -g -O0 -fPIC -shared-intel -c test_pulse_create.cpp
You should see the very same error message as above - Blitz -pthread error
Solution
After a long search a solution has been found, however it is not very elegant. Requires editing pkg-config for Blitz, which by default is prepared by SPACK.
The change is subtle and affects only CFLAGS
which should look like this:
Code Block |
---|
Cflags: -I${includedir} -pthread |
Below is a before and after comparison:
Vanilla Blitz's pkg-config - provided by SPACK | Changed pkg-config for Blitz | ||||
---|---|---|---|---|---|
|
|
The above solution acts the same way as adding the -pthread
flag explicitly to the compilation command in point 6 at 2.1 How to reproduce issue, something like below:
Code Block |
---|
icpc `pkg-config blitz --cflags` -I/marconi_work/FUSIO_HLST/msmukows/IMAS_INSTALLER/intel_installer/src/3.36.0/ual/4.10.1/cppinterface/src -I/marconi_work/FUSIO_HLST/msmukows/IMAS_INSTALLER/intel_installer/src/3.36.0/ual/4.10.1/lowlevel -g -O0 -fPIC -shared-intel -c -pthread test_pulse_create.cpp |