1. Packaging schema

Actors will be packaged using pip based approach. This way it will be possible to:

  • install actors centrally
  • install actors inside virtual environment
  • host private PIP based repository

This section describes how packaging of an actor will look like and what will be the final structure of distributable package.

2. Sample code

Sample code (below) illustrates the way of embedding Fortran code inside Python. We start with very simple Fortran routine (distance). As we intent to wrap the code inside Python, we have to provide iso_c_binding based code as well (this code will be called from Python). In final solution this code will be autogenerated.

module distance_module
  use iso_c_binding

  contains

    subroutine distance_wrapper(v, t, d) bind(c, name="distance_wrapper")

      real (c_float), intent(inout)  :: v
      real (c_float), intent(inout)  :: t
      real (c_float), intent(inout)  :: d

      call distance(v, t, d)

    end subroutine

    subroutine distance(v, t, d)

      implicit none

      real, intent(in)  :: v
      real, intent(out) :: t
      real, intent(out) :: d

      d = v * t

    end subroutine

end module distance_module

What we eventually want to do is to be able to call something like this inside Python

from distance_wrapper.actor import distance

print('Distance: ', distance( 20.0, 2.0 ))

Even though in a sample project all the codes are part of the repo, in final solution most of the elements will be autogenerated. The only code (provided by user) will be Fortran one.

Sample application with two codes based scenario can be found here: https://github.com/mkopsnc/marmot

In order to run samples, simply follow instructions inside README.md file

3. Sample structure

Sample code has following structure

distance                               - for the convenience all the code is in one dir
|-- distance_c_wrapper                 - C based wrapper - layer that allows to call
|   |-- Makefile                         Fortran code from Python
|   |-- include
|   |   `-- distance_c_wrapper.h         Note! All the code inside distance_c_wrapper
|   `-- src                              will be autogenerated. Shape of the code will depend
|       |-- distancewrapper              on project definition (description of what is supposed
|       |   `-- distance_c_wrapper.c     to be generated)
|       `-- main.c
|-- distance_fortran                   - This is the only code that will be provided by user
|   |-- Makefile                         Result of the compilation is a lib##.a file that
|   `-- src                              is later on packed inside shared lib that is called
|       |-- distance                     from Python
|       |   `-- distance.F90
|       `-- main.F90
`-- distance_python                    - Python based wrapper. This is the code that will
    |-- MANIFEST.in                      be incorporated inside Python scripts/Python workflows
    |-- README.md
    |-- distance_wrapper
    |   |-- __init__.py                  Note! This code will be autogenerated (see above)
    |   `-- actor.py
    |-- setup.cfg
    `-- setup.py

4. Benefits of this approach

This approach allows to use the code in numerous scenarios

4.1. PYTHONPATH

It is possible to use the code directly inside script by simply exporting location of the code using PYTHONPATH.

> export PYTHONPATH=`pwd`/distance_python:${PYTHONPATH}

4.2. PIP install

Current structure allows to install wrapper inside central installation of Python or inside virtual environment. All we have to do is to use pip install. 

> pip3 install --upgrade --force-reinstall `pwd`/distance_python

4.3. PIP repository

With this structure it is possible to generate compressed versions of pip packages and distribute them via custom, self hosted, pip repository.

5. References

  1. pip documentation: https://pip.pypa.io/en/stable/
  2. Packaging Python Projects: https://packaging.python.org/tutorials/packaging-projects/
  3. Packaging Python Projects - Hosting your own simple repository: https://packaging.python.org/guides/hosting-your-own-index/
  4. ISO_C_BINDING: https://gcc.gnu.org/onlinedocs/gfortran/ISO_005fC_005fBINDING.html
  • No labels