Versions Compared

Old Version 2

changes.mady.by.user Bartosz Palak

Saved on

compared with

New Version Current

changes.mady.by.user Bartosz Palak

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents
1 Introduction

1.1

Libraries layout

CPT May 2018

Scope

Establish a standard method for laying out the common libraries to be used by components in a workflow. The complexity lies in the fact that components may be compiled with different library versions and even different compilers and versions of them, so the library layout must enable dynamically loading the relevant library for each workflow component.

1.2 Background and decision status

  • The previous method used by the “ITM” is judged too complicate to maintain in the long term, so some adaptation should be made at the occasion of the transition to IMAS.

...

  • IO provides only a framework where local sysadmins add their own path extensions to the user environment, because of no control on how other dependencies will be resolved on future systems (see discussions with IO on this topic: IMAS-1188, IMAS-1484, IMAS-1072, IMAS-1888),

Taking these factors into consideration CPT proposes "EUROfusion flavor" of library modules and related layout

Libraries layout

Examples are provided for a single library (XMLLIB) for the sake of clarity, to be generalised for all libraries.

In case the library depends on the Data Dictionary (DD) version, this DD version information shall be included in the <LIB_VERSION> string.

.

2 Assumptions

2.1 Usage scenario:

  1. User defines compiler being used and loads all IMAS/EF DD not dependent libraries / tools

    module load wpcdenv[/<compiler_vendor>/<compiler_version>/<version of this module>]

  2. User load imas module plus all components dependent on IMAS/Data Dictionary
    module load imasenv[/<imas_ver>/ual/<engine_ver>/<version of this module>]

  3. User cherry-picks version of library
    module switch <imas_lib>

2.2 Assumptions

  • imaslibs and imasenv modules have to be loaded prior to <imas_lib>

  • module <imas_lib> can be only switched

3 Modules and packages

3.1 Module names

...

The module name explicitly identifies the library’s name and version and the compiler’s name and version.

Code Block
<ROOT_MODULES_PATH>/module switch <LIB_NAME>/<LIB_VERSION>-/<COMPILER_NAME>-/<COMPILER_VERSION>

...

/imas/<DD_VER>
E.g.:
module switch ggd/1.8.0[/intel/17/imas/3.21.0/]

 

Loading the module is thus done via:

Code Block
module load xmllib/1.0-ifort-17

A module adds a single directory to the $PKG_CONFIG_PATH (making a single version available at compile time) and adds <ROOT_LIB_PATH>/<LIB_NAME>/lib in $LD_LIBRARY_PATH (making all versions available at run time for dynamically loading the library, see below).

  • Each module is conflicting with the others (of the same library).
  • Meta-modules allowing loading at once all modules in a coherent environment for a given workflow/application (if not conflicting) will be available.
  • In case a complex workflow/application contains components using conflicting libraries (i.e. using different versions/compilers for the same library), the relevant module must be loaded (switched) prior to the component compilation, and then unloaded.
Info
titleNaming convention for compilers
GNU Compiler Collection -> gcc
Intel           		-> intel

 

 

3.2 Package names

Code Block
pkg-config --cflags --libs <LIB_NAME>
E.g.:
pkg-config --cflags --libs ggd

 

  • Please notice that there are no longer <LIB_NAME>-<COMPILER_NAME> (e.g. ggd-ifort packages), as given module could be loaded for one compiler only

 

4. RPATH - library discovery at runtime

Actors that consist any workflow can be compiled with different versions of particular library. To allow an actor to resolve all dependent libraries at runtime,

every package of the library, beside providing info about includes (*.h, *.mod) and library (*.so, *.a) has to provide RPATH.

Code Block
languagebash
>> pkg-config --cflags --libs <LIB_NAME>
>> -Iinclude -Llib -lshared -Wl,-rpath,$(CURDIR)/lib

This way, each code at runtime will refer to version of library used for linking.

...

ProsCons
  • Different codes can use different versions of libraries at runtime
  • Each library is released as a separate installation (it's easy to maintain each version separately)
  • No long LD_LIBRARY_PATH variables
  • LD_LIBRARY_PATH has lower precedence than rpath
  • It is still possible to use LD_PRELOAD to enforce given library to be used (e.g. debugging purposes)
  • It's not possible to move libraries from their original location
  • It's not possible to point to any other (even backward compatible) version
  • To use a new release of the library, user's code must be recompiled

5 Library directory layout

 

...

The library layout explicitly identifies the library’s name and version and the compiler’s name and version and puts the various files of the library as follows:

Code Block
<ROOT_LIB_PATH>/<LIB_NAME>/<LIB_VERSION>-/<COMPILER_NAME>-/<COMPILER_VERSION>/imas/<DD_VER>/include/*.h
<ROOT_LIB_PATH>/<LIB_NAME>/<LIB_VERSION>-/<COMPILER_NAME>-/<COMPILER_VERSION>/imas/<DD_VER>/include/*.mod
<ROOT_LIB_PATH>/<LIB_NAME>/<LIB_VERSION>-/<COMPILER_NAME>-/<COMPILER_VERSION>/imas/<DD_VER>/lib/*.a
<ROOT_LIB_PATH>/<LIB_NAME>/<LIB_VERSION>-/<COMPILER_NAME>-/<COMPILER_VERSION>/imas/<DD_VER>/lib/*.so
<ROOT_LIB_PATH>/<LIB_NAME>/<LIB_VERSION>-/<COMPILER_NAME>-/<COMPILER_VERSION>/imas/<DD_VER>/lib/pkgconfig/<LIB_NAME>.pc
<ROOT_LIB_PATH>/<LIB_NAME>/<LIB_VERSION>-/<COMPILER_NAME>-/<COMPILER_VERSION>/src
<ROOT_LIB_PATH>/<LIB_NAME>/<LIB_VERSION>-<COMPILER_NAME>-<COMPILER_VERSION>/exampleimas/<DD_VER>/src
<ROOT_LIB_PATH>/<LIB_NAME>/<LIB_VERSION>-/<COMPILER_NAME>-/<COMPILER_VERSION>/man

Note that all pc files have the same name, for the sake of simplicity (for the code developer).

In addition, a “central” repository

Code Block
<ROOT_LIB_PATH>/<LIB_NAME>/lib

 contains links to the *.so files (for all library versions and compiler names and versions), in order to keep $LD_LIBRARY_PATH length below the operating system limit.

Code Block
imas/<DD_VER>/example
<ROOT_LIB_PATH>/<LIB_NAME>/lib/lib<LIB_NAME>-<LIB_VERSION>-/<COMPILER_NAME>-<COMPILER_VERSION>.so -> ../<LIB_VERSION>-<COMPILER_NAME>-<COMPILER_VERSION>/libimas/lib<LIB_NAME>.s

Example

Code Block
<ROOT_LIB_PATH>/xmllib/1.0-ifort-17/include/*.h *.mod 
<ROOT_LIB_PATH>/xmllib/1.0-ifort-17/lib/*.a *.so
<ROOT_LIB_PATH>/xmllib/1.0-ifort-17/lib/pkgconfig/xmllib.pc
<ROOT_LIB_PATH>/xmllib/1.0-ifort-17/src
<ROOT_LIB_PATH>/xmllib/1.0-ifort-17/example
<ROOT_LIB_PATH>/xmllib/1.0-ifort-17/man
<ROOT_LIB_PATH>/xmllib/lib/libxmllib-1.0-ifort-17.so  -> ../1.0-ifort-17/lib/libxmllib.so

Suggested naming convention for modules that depend on compiler

Code Block
// C code
GNU C           -> gnu
Intel           -> intel
 
// Fortran
Intel           -> intel
GNU Fortran     -> gnu
G95             -> g95
 

 

Example of XMLLIB installation based on IMAS based release

<DD_VER>/man

 

 

 

Code Block
titleGGD: Example of library dir layout
ggd
`-- 1.8.0
    |-- gcc
    |   `-- 6
    |       `-- imas
Code Block
swimas
|-- etc
|   `-- modulefiles
|       `-- xmllib
|           |-- 4.0-gnu-6.1.0
|           `-- 4.0-intel-17.0.4
`-- extra
    `-- xmllib
        |-- 4.0-gnu-4.8.5
        |   |-- include
        |   |   |-- f90_file_reader.mod
        |   |   |-- mod_f90_kind.mod
        |   |   |`-- string_manipulation_tools.mod
        |3.21.0
    |   |-- xml2eg_mdl.mod
        |   |   |-- xml_file_reader.mod
 include
       |   |   |-- xmllib_parser.mod
        |   |   |-- xmllibids_pathqueryassert.mod
        |   |   `-- xmllib_types.mod
        |   `-- lib
        |       |-- libxmllib.a
   ****
     |       `-- pkgconfig
        |           `-- xmllib.pc
        |-- 4.0-intel-17.0.4
        |   |-- include
        |   |   |-- f90_file_reader.mod
        |   |   |-- mod_f90_kindids_utility.mod
        |   |   |-- string_manipulation_tools.mod
        |   |   |`-- xml2eg_mdl.mod
    lib
    |   |   |-- xml_file_reader.mod
        |   |   |-- xmllib_parserlibggd.moda
        |   |   |-- xmllib_pathquery.mod
        |   |   `-- xmllib_types.modpkgconfig
        |   `-- lib
        |       |-- libxmllib.a
        |       `-- pkgconfig
        |           `-- xmllibggd.pc
        `-- lib                                                                        # this can be completely removed if we start using -rpath
   intel
         |-- libxmllib-4.0-gnu-4.8.5.a -> ../4.0-gnu-4.8.5/lib/libxmllib.a          # this can be completely removed if we start using -rpath
            `-- libxmllib-4.0-intel-17.0.4.a -> ../4.0-intel-17.0.4/lib/libxmllib.a    # this can be completely removed if we start using -rpath

RPATH - compiling codes with fixed location of libraries

One of the drawbacks of the module based layout is the problem of pointing to proper library (if we don't have symbolic links with fully qualified names).

This issue can be solved with rpath while compiling the code. Instead of having number of libraries, each with slightly different name, we can use rpath instead.

Code Block
languagebash
gcc -o $@ $< -Iinclude -Llib -lshared -Wl,-rpath,$(CURDIR)/lib

This way, each code will refer to version of library used for linking.

ProsCons
different codes can use different versions of librarieswith each release of new version of the library user's code must be recompiled
each library is released as a separate installation (it's easy to maintain each version separately)it's not possible to move libraries from their original location
we are not forced to build log LD_LIBRARY_PATH variables 
LD_LIBRARY_PATH has lower precedence than rpath 
It is still possible to use LD_PRELOAD to enforce given library to be used (e.g. debugging purposes) 

RPATH - slightly different layout of directories

In case of rpath based compilation we don't need centrally located lib directory at all. All files (paths) will be provided by pc files for each and every module.

Code Block
swimas
|-- etc
|   `-- modulefiles
|       `-- xmllibimas
|           |-- 4.0-gnu-6.1.0
|           `-- 4.0-intel-17.0.4
`-- extra3.21.0
    `-- xmllib
        |-- 4.0-gnu-4.8.5
        |   |-- include
        |   |   |-- f90_file_reader.mod
        |   |   |-- modids_f90_kindassert.mod
        |   |   |-- string_manipulation_tools.mod
        |   |   |-- xml2eg_mdl.mod****
        |   |   |-- xml_file_reader.mod
        |   |   |`-- xmllibids_parserutility.mod
        |   |   |-- xmllib_pathquery.mod
        |   |   `-- xmllib_types.modlib
        |   `-- lib
        |       |-- libxmlliblibggd.a
        |       `-- pkgconfig
        |           `-- xmllib.pc
        `-- 4.0-intel-17.0.4
            |-- include
            |   |-- f90_file_reader.mod
            |   |-- mod_f90_kind.mod
            |   |-- string_manipulation_tools.mod
            |   |-- xml2eg_mdl.mod
            |   |-- xml_file_reader.mod
            |   |-- xmllib_parser.mod`-- pkgconfig
            |   |-- xmllib_pathquery.mod
            |   `-- xmllib_types.mod
            `-- lib
                |-- libxmllib.a
                `-- pkgconfig
                    `-- xmllib.pc
ggd.p

4.