...
| Table of Contents |
|---|
Introduction
Motivations
Complex simulations often combines a number of physics codes, potentially provided by various actors and written in different programming languages. To make them working together, an additional layer, that 'orchestrates' execution of particular codes, and takes care on passing data between 'producers' and 'consumers' is needed. Sometimes the functionality of such layer is provided by dedicated software (aka 'workflow orchestrators', like Kepler https://kepler-project.org/), sometimes it can be handled by much simpler mechanism like Python scripts. Unfortunately all components ('actors') that constitute a computing scenario ('workflow') must be implemented in the same programming language, defining the same API.
Unfortunately, in most cases, scientific, simulation codes that performs computing intensive calculations (due to performance reasons) are written in C++ or Fortran, while 'workflow orchestrators' are implemented in (more popular nowadays) languages, like Java, Python, etc. Hence the need for a 'wrapper' that intermediates between native code language and language of the orchestrator. Such wrappers can be developed manually, however users may benefit from a tool that automatise this process - iWrap
iWrap - actor generator
iWrap is a modular component generator, implemented in Python, used for creating IMAS actors from physics models. This mechanism allows to integrate physics codes written in one language (Fortran, CPP) within complex computing scenarios designed in other language (e.g. Python).
It's plug-in based modular design with clear separation of concerns allows to generate various types of actors and easily change data access paradigm (from dataset descriptor for AL to direct HDC data for instance)
For user conveniency it provides two kinds of interfaces: user friendly graphical interface that allows non-experienced users to define an actor in intuitive way and command line interface foreseen for more advanced users that may want to e.g. automatise actor generation process using scripts.
- iWrap generates a Fortran/CPP wrapper, which intermediates between Kepler actor and user code in terms of:
- reading/writing of in/out physical data (IDS)
- passing other arguments to/from the actor
- iWrap creates a Python script (aka an actor) that:
- calls a user code
- provides error handling
- calls debugger (if run in "debug" mode)
...
Glossary
Scenario (aka workflow)
- A set of components (actors) constituting a directed graph to execute a computing algorithm
- Actors are dependent: a control and data is passed from actor to actor
- Usually the order of actors execution and the way how data are passed from actor to actor is managed by so called "workflow system". Such manager can be a simple script (codes) or more sophisticated "orchestrator" (e.g. Kepler)
Actor
- A basic component of scenario / workflow
- An actor performs some actions (e.g. computations, visualisation, etc)
- Usually given actor consumes results provided by a previous actor in a scenario and produces data for a next actor in a scenario
Actor API strictly depends on targeted workflow system: an orchestrator "fires" particular actions on actor
- An actor, using its internal mechanisms ('wrappers') calls 'native code' method(s), usually written in other language than an actor
Native code
| Info |
|---|
Glossary Scenario (aka workflow)
Actor
Native code
|
Motivations
Complex simulations often combines a number of physics codes, potentially provided by various actors and written in different programming languages. To make them working together, an additional layer, that 'orchestrates' execution of particular codes, and takes care on passing data between 'producers' and 'consumers' is needed. Sometimes the functionality of such layer is provided by dedicated software (aka 'workflow orchestrators', like Kepler https://kepler-project.org/), sometimes it can be handled by much simpler mechanism like Python scripts. Unfortunately all components ('actors') that constitute a computing scenario ('workflow') must be implemented in the same programming language, defining the same API.
Unfortunately, in most cases, scientific, simulation codes that performs computing intensive calculations (due to performance reasons) are written in C++ or Fortran, while 'workflow orchestrators' are implemented in (more popular nowadays) languages, like Java, Python, etc. Hence the need for a 'wrapper' that intermediates between native code language and language of the orchestrator. Such wrappers can be developed manually, however users may benefit from a tool that automatise this process - iWrap
iWrap - actor generator
iWrap is a modular component generator, implemented in Python, used for creating IMAS actors from physics models. This mechanism allows to integrate physics codes written in one language (Fortran, CPP) within complex computing scenarios designed in other language (e.g. Python).
It's plug-in based modular design with clear separation of concerns allows to generate various types of actors and easily change data access paradigm (from dataset descriptor for AL to direct HDC data for instance)
For user conveniency it provides two kinds of interfaces: user friendly graphical interface that allows non-experienced users to define an actor in intuitive way and command line interface foreseen for more advanced users that may want to e.g. automatise actor generation process using scripts.
- iWrap generates a Fortran/CPP wrapper, which intermediates between Kepler actor and user code in terms of:
- reading/writing of in/out physical data (IDS)
- passing other arguments to/from the actor
- iWrap creates a Python script (aka an actor) that:
- calls a user code
- provides error handling
- calls debugger (if run in "debug" mode)
...
Preparation of code
A signature of user code must follow strict rules to allow interaction between it and wrapping actor. Please use following >>link<< to get detailed guidelines for integration of native code into workflows using iWrap
...
compiler:- meaning: the name/vendor of the compiler (and not compiler command!) used to compile native codes
- value: string, one of vendors of compilers, currently: 'Intel' or 'GCC'
- example: 'Intel'
mpi_flavour- meaning: MPI compiler flavour to be used
- values: string, one of: MPICH, MPICH2, MVAPICH2, OpenMPI, etc.
- example 'MPICH2'
open_mp:- meaning: if user code should be compiled with OpenMP flag
- values: boolean
- example 'true'
system_libraries:- meaning: a list of system libraries, managed using pkg-config mechanism, that has to be used while native code linking
- value: a list of system libraries names, as they are published by pkg-config
example:
- fftw3f- glib- mkl
custom_libraries:- meaning: additional libraries, not managed by pkg-config mechanism, necessary to link of the physics code :
- value: a list of paths to libraries
example:
- ./lib/custom/libcustom1.a- ./lib/custom/libcustom2.a
Example - Fortran code description
...