...
- User defines compiler being used and loads all IMAS/EF DD not dependent libraries / tools
module load imaslibs[/<compiler_vendor>/<compiler_version>/<version of this module>] - User load
imas moduleplus all components dependent on IMAS/Data Dictionarymodule load imasenv[/<imas_ver>/ual/<engine_ver>/<version of this module>] - User cherry-picks version of library
module switch <imas_lib>
| Warning | ||
|---|---|---|
| ||
|
Libraries layout
Examples are provided for a single library (XMLLIB) for the sake of clarity, to be generalised for all libraries.
In case the library depends on the Data Dictionary (DD) version, this DD version information shall be included in the <LIB_VERSION> string.
Modules
The module name explicitly identifies the library’s name and version and the compiler’s name and version.
Module names
The module name explicitly identifies the library’s name and version and the compiler’s name and version.
| Code Block |
|---|
module switch <LIB_ |
| Code Block |
<ROOT_MODULES_PATH>/<LIB_NAME>/<LIB_VERSION>-<COMPILER_NAME>-<COMPILER_VERSION> |
...
Loading the module is thus done via:
| Code Block |
|---|
module load xmllib/1.0-ifort-17 |
A module adds a single directory to the $PKG_CONFIG_PATH (making a single version available at compile time) and adds <ROOT_LIB_PATH>/<LIB_NAME>/lib in $LD_LIBRARY_PATH (making all versions available at run time for dynamically loading the library, see below).
...
E.g.:
module switch ggd/1.8.0[/intel/17/imas/3.21.0/ual/3.8.4] |
- Each module is conflicting with the others (of the same library).
- Meta-modules allowing loading at once all modules in a coherent environment for a given workflow/application (if not conflicting) will be available.
- In case a complex workflow/application contains components using conflicting libraries (i.e. using different versions/compilers for the same library), the relevant module must be loaded (switched) prior to the component compilation, and then unloaded.
Libraries
The library layout explicitly identifies the library’s name and version and the compiler’s name and version and puts the various files of the library as follows:
| Info | ||
|---|---|---|
| ||
GNU Compiler Collection -> gcc
Intel -> intel
|
Package names
| Code Block |
|---|
pkg-config --cflags --libs <LIB_NAME>
E.g.:
pkg-config --cflags --libs ggd |
- Please notice that there are no longer <LIB_NAME>-<COMPILER_NAME> (e.g. ggd-ifort packages), as given module could be loaded for one compiler only
RPATH - library discovery at runtime
One of the drawbacks of the module based layout is the problem of pointing to proper library (if we don't have symbolic links with fully qualified names).
This issue can be solved with rpath while compiling the code. Instead of having number of libraries, each with slightly different name, we can use rpath instead.
| Code Block | ||
|---|---|---|
| ||
gcc -o $@ $< -Iinclude -Llib -lshared -Wl,-rpath,$(CURDIR)/lib |
This way, each code will refer to version of library used for linking.
| Pros | Cons |
|---|---|
| different codes can use different versions of libraries | with each release of new version of the library user's code must be recompiled |
| each library is released as a separate installation (it's easy to maintain each version separately) | it's not possible to move libraries from their original location |
| we are not forced to build log LD_LIBRARY_PATH variables | |
| LD_LIBRARY_PATH has lower precedence than rpath | |
| It is still possible to use LD_PRELOAD to enforce given library to be used (e.g. debugging purposes) |
Libraries layout
Examples are provided for a single library (XMLLIB) for the sake of clarity, to be generalised for all libraries.
In case the library depends on the Data Dictionary (DD) version, this DD version information shall be included in the <LIB_VERSION> string.
Libraries
The library layout explicitly identifies the library’s name and version and the compiler’s name and version and puts the various files of the library as follows:
| Code Block |
|---|
| Code Block |
<ROOT_LIB_PATH>/<LIB_NAME>/<LIB_VERSION>-<COMPILER_NAME>-<COMPILER_VERSION>/include/*.h <ROOT_LIB_PATH>/<LIB_NAME>/<LIB_VERSION>-<COMPILER_NAME>-<COMPILER_VERSION>/include/*.mod <ROOT_LIB_PATH>/<LIB_NAME>/<LIB_VERSION>-<COMPILER_NAME>-<COMPILER_VERSION>/lib/*.a <ROOT_LIB_PATH>/<LIB_NAME>/<LIB_VERSION>-<COMPILER_NAME>-<COMPILER_VERSION>/lib/*.so <ROOT_LIB_PATH>/<LIB_NAME>/<LIB_VERSION>-<COMPILER_NAME>-<COMPILER_VERSION>/lib/pkgconfig/<LIB_NAME>.pc <ROOT_LIB_PATH>/<LIB_NAME>/<LIB_VERSION>-<COMPILER_NAME>-<COMPILER_VERSION>/include/src*.h <ROOT_LIB_PATH>/<LIB_NAME>/<LIB_VERSION>-<COMPILER_NAME>-<COMPILER_VERSION>/include/example*.mod <ROOT_LIB_PATH>/<LIB_NAME>/<LIB_VERSION>-<COMPILER_NAME>-<COMPILER_VERSION>/man |
Note that all pc files have the same name, for the sake of simplicity (for the code developer).
In addition, a “central” repository
| Code Block |
|---|
/lib/*.a <ROOT_LIB_PATH>/<LIB_NAME>/lib |
contains links to the *.so files (for all library versions and compiler names and versions), in order to keep $LD_LIBRARY_PATH length below the operating system limit.
| Code Block |
|---|
<LIB_VERSION>-<COMPILER_NAME>-<COMPILER_VERSION>/lib/*.so <ROOT_LIB_PATH>/<LIB_NAME>/<LIB_VERSION>-<COMPILER_NAME>-<COMPILER_VERSION>/lib/pkgconfig/<LIB_NAME>.pc <ROOT_LIB_PATH>/<LIB_NAME>/lib/lib<LIB_NAME>-<LIB_VERSION>-<COMPILER_NAME>-<COMPILER_VERSION>/src <ROOT_LIB_PATH>/<LIB_NAME>/<LIB_VERSION>-<COMPILER_NAME>-<COMPILER_VERSION>.so -> ../example <ROOT_LIB_PATH>/<LIB_NAME>/<LIB_VERSION>-<COMPILER_NAME>-<COMPILER_VERSION>/lib/lib<LIB_NAME>.s |
Example
man |
Note that all pc files have the same name, for the sake of simplicity (for the code developer).
In addition, a “central” repository
| Code Block |
|---|
| Code Block |
<ROOT_LIB_PATH>/xmllib/1.0-ifort-17/include/*.h *.mod <ROOT_LIB_PATH>/xmllib/1.0-ifort-17/lib/*.a *.so <LIB_NAME>/lib |
contains links to the *.so files (for all library versions and compiler names and versions), in order to keep $LD_LIBRARY_PATH length below the operating system limit.
| Code Block |
|---|
<ROOT_LIB_PATH>/<LIB_NAME>/lib/lib<LIB_NAME>-<LIB_VERSION>-<COMPILER_NAME>-<COMPILER_VERSION>.so -> ../<LIB_VERSION>-<COMPILER_NAME>-<COMPILER_VERSION>/lib/lib<LIB_NAME>.s |
Example
| Code Block |
|---|
xmllib/1.0-ifort-17/lib/pkgconfig/xmllib.pc <ROOT_LIB_PATH>/xmllib/1.0-ifort-17/src/include/*.h *.mod <ROOT_LIB_PATH>/xmllib/1.0-ifort-17/example/lib/*.a *.so <ROOT_LIB_PATH>/xmllib/1.0-ifort-17/man/lib/pkgconfig/xmllib.pc <ROOT_LIB_PATH>/xmllib/lib/libxmllib-1.0-ifort-17.so -> ../xmllib/1.0-ifort-17/lib/libxmllib.so |
Suggested naming convention for modules that depend on compiler
| Code Block |
|---|
// C code GNU C -> gnu Intel -> intel // Fortran Intel -> intel GNU Fortran -> gnu G95 -> g95 src <ROOT_LIB_PATH>/xmllib/1.0-ifort-17/example <ROOT_LIB_PATH>/xmllib/1.0-ifort-17/man <ROOT_LIB_PATH>/xmllib/lib/libxmllib-1.0-ifort-17.so -> ../1.0-ifort-17/lib/libxmllib.so |
Example of XMLLIB installation based on IMAS based release
...
| Code Block |
|---|
swimas
|-- etc
| `-- modulefiles
| `-- xmllib
| |-- 4.0-gnu-6.1.0
| `-- 4.0-intel-17.0.4
`-- extra
`-- xmllib
|-- 4.0-gnu-4.8.5
| |-- include
| | |-- f90_file_reader.mod
| | |-- mod_f90_kind.mod
| | |-- string_manipulation_tools.mod
| | |-- xml2eg_mdl.mod
| | |-- xml_file_reader.mod
| | |-- xmllib_parser.mod
| | |-- xmllib_pathquery.mod
| | `-- xmllib_types.mod
| `-- lib
| |-- libxmllib.a
| `-- pkgconfig
| `-- xmllib.pc
`-- 4.0-intel-17.0.4
|-- include
| |-- f90_file_reader.mod
| |-- mod_f90_kind.mod
| |-- string_manipulation_tools.mod
| |-- xml2eg_mdl.mod
| |-- xml_file_reader.mod
| |-- xmllib_parser.mod
| |-- xmllib_pathquery.mod
| `-- xmllib_types.mod
`-- lib
|-- libxmllib.a
`-- pkgconfig
`-- xmllib.pc
|