You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 8 Next »

1.1.

1.2. Scope

Establish a standard method for laying out the common libraries to be used by components in a workflow. The complexity lies in the fact that components may be compiled with different library versions and even different compilers and versions of them, so the library layout must enable dynamically loading the relevant library for each workflow component.

1.3. Background and decision status

  • The previous method used by the “ITM” is judged too complicate to maintain in the long term, so some adaptation should be made at the occasion of the transition to IMAS.
  • IO provides only a framework where local sysadmins add their own path extensions to the user environment, because of no control on how other dependencies will be resolved on future systems (see discussions with IO on this topic: IMAS-1188, IMAS-1484, IMAS-1072),

Taking these factors into consideration CPT proposes "EUROfusion flavor" of library modules and related layout.

 

1.4. Usage scenario:

  1. User defines compiler being used and loads all IMAS/EF DD not dependent libraries / tools

     

    module load imaslibs[/<compiler_vendor>/<compiler_version>/<version of this module>]

  2. User load imas module plus all components dependent on IMAS/Data Dictionary
    module load imasenv[/<imas_ver>/ual/<engine_ver>/<version of this module>]

  3. User cherry-picks version of library
    module switch <imas_lib>

Assumptions

  • imaslibs and imasenv modules have to be loaded prior to <imas_lib>

  • module <imas_lib> can be only switched

 

1.5. Module names

The module name explicitly identifies the library’s name and version and the compiler’s name and version.

module switch <LIB_NAME>/<LIB_VERSION>/<COMPILER_NAME>/<COMPILER_VERSION>/imas/<DD_VER>/ual/<ENGINE_VER>
E.g.:
module switch ggd/1.8.0[/intel/17/imas/3.21.0/ual/3.8.4]

 

  • Each module is conflicting with the others (of the same library).
  • Meta-modules allowing loading at once all modules in a coherent environment for a given workflow/application (if not conflicting) will be available.
  • In case a complex workflow/application contains components using conflicting libraries (i.e. using different versions/compilers for the same library), the relevant module must be loaded (switched) prior to the component compilation, and then unloaded.

Naming convention for compilers

GNU Compiler Collection -> gcc
Intel           		-> intel

 

 

1.6. Package names

pkg-config --cflags --libs <LIB_NAME>
E.g.:
pkg-config --cflags --libs ggd

 

  • Please notice that there are no longer <LIB_NAME>-<COMPILER_NAME> (e.g. ggd-ifort packages), as given module could be loaded for one compiler only

 

1.7. RPATH - library discovery at runtime

Actors that consist any workflow can be compiled with different versions of particular library. To allow an actor to resolve all dependent libraries at runtime,

every package of the library, beside providing info about includes (*.h, *.mod) and library (*.so, *.a) has to provide RPATH.

>> pkg-config --cflags --libs <LIB_NAME>
>> -Iinclude -Llib -lshared -Wl,-rpath,$(CURDIR)/lib

This way, each code at runtime will refer to version of library used for linking.

ProsCons
Different codes can use different versions of libraries at runtime

It's not possible to move libraries from their original location

Each library is released as a separate installation (it's easy to maintain each version separately) It's not possible to point to any other (even backward compatible) version
No long LD_LIBRARY_PATH variablesTo use a new release of the library, user's code must be recompiled
LD_LIBRARY_PATH has lower precedence than rpath 
It is still possible to use LD_PRELOAD to enforce given library to be used (e.g. debugging purposes)

2. TO BE ADDED

2.1. Library directory/module layout

 

The library layout explicitly identifies the library’s name and version and the compiler’s name and version and puts the various files of the library as follows:

<ROOT_LIB_PATH>/<LIB_NAME>/<LIB_VERSION>/<COMPILER_NAME>/<COMPILER_VERSION>/imas/<DD_VER>/ual/<ENGINE_VER>/include/*.h
<ROOT_LIB_PATH>/<LIB_NAME>/<LIB_VERSION>/<COMPILER_NAME>/<COMPILER_VERSION>/imas/<DD_VER>/ual/<ENGINE_VER>/include/*.mod
<ROOT_LIB_PATH>/<LIB_NAME>/<LIB_VERSION>/<COMPILER_NAME>/<COMPILER_VERSION>/imas/<DD_VER>/ual/<ENGINE_VER>/lib/*.a
<ROOT_LIB_PATH>/<LIB_NAME>/<LIB_VERSION>/<COMPILER_NAME>/<COMPILER_VERSION>/imas/<DD_VER>/ual/<ENGINE_VER>/lib/*.so
<ROOT_LIB_PATH>/<LIB_NAME>/<LIB_VERSION>/<COMPILER_NAME>/<COMPILER_VERSION>/imas/<DD_VER>/ual/<ENGINE_VER>/lib/pkgconfig/<LIB_NAME>.pc
<ROOT_LIB_PATH>/<LIB_NAME>/<LIB_VERSION>/<COMPILER_NAME>/<COMPILER_VERSION>/imas/<DD_VER>/ual/<ENGINE_VER>/src
<ROOT_LIB_PATH>/<LIB_NAME>/<LIB_VERSION>/<COMPILER_NAME>/<COMPILER_VERSION>/imas/<DD_VER>/ual/<ENGINE_VER>/example
<ROOT_LIB_PATH>/<LIB_NAME>/<LIB_VERSION>/<COMPILER_NAME>/<COMPILER_VERSION>/imas/<DD_VER>/ual/<ENGINE_VER>/man
  • No labels