...

SUMI depends on two powerfull libraries to perform its tasks. These are Paramiko (data transfer) and SAGA (remote job submission). To install the dependencies we need to download python libraries, but given that we do not have root permissions we will make usage of a virtual environment which will allow to install these libraries locally. For this purpose we will use "virtualenv". Virtualenv creates a python environment in local folders and allows to create a virtual python environment where we can install libraries locally. To set it up we run the following commands:

    mkdir sumi-virtualenv

    virtualenv sumi-virtualenv

...

In the case of TCSH shells (as Gateway)

    source sumi-virtualenv/bin/activate.csh

In the case of Bash shells (most used shells)

    source sumi-virtualenv/bin/activate

Our terminal prompt will now show the folder name in front of out username in the following way:

    [sumi-virtualenv] <g2user@s65

To install the dependencies, now we can run the "pip" command which will install the python libraries in our local virtualenv

    pip install saga-python==0.50.01 paramiko

Once the dependencies have been installed, we can download and configure SUMI. To retrieve the code, clone the code from the repository

    git clone https://albertbsc@bitbucket.org/albertbsc/sumi.git

...

For TCSH shell systems (as Gateway)

    setenv PATH $PATH\:$HOME/sumi/bin

For Bash shells

    export PATH=$PATH:$PWD/sumi/bin/

SUMI requires two configuration job files which contain the information of the jobs to be sumitted and the HPC cluster where we are going to submit. For this we need to create the configuration folder and copy the configuration files jobs.conf and servers.conf from sumi/conf/ directory.

    mkdir $HOME/.sumi

    cp sumi/conf/*.conf $HOME/.sumi

Now, we are ready to run SUMI. Execute the option "-h" to see all the

sumi -h

usage: sumi.py [-h] [-r] [-u UP] [-d DOWN] [-m MACHINE [MACHINE ...]]

               [-j JOBS [JOBS ...]]

Submission Manager for IMAS

optional arguments:

  -h, --help            show this help message and exit

  -r, --run             Run the configured job on a specific cluster

  -u UP, --upload UP    Upload local file

  -d DOWN, --download DOWN

                        Download remote file

  -m MACHINE [MACHINE ...], --machine MACHINE [MACHINE ...]

                        List of machines were to submit

  -j JOBS [JOBS ...], --job JOBS [JOBS ...]

                        List of jobs to be submitted

Then, SUMI has been installed successfully

Configure sample job and big Marconi connection

To configure a job we have to edit the files copied on the

Test connection with Marconi

We will make usage of uDocker. uDocker is based on Docker but it does not require root permissions. Therefore, it does not have some features as TCP/IP sniffing or other action that require root privileges. uDocker will allow to simulate an environment with root permissions (UID = 0) and install IMAS inside.

Therefore, the very first step is installing uDocker. uDokcer will be set up once and used many times

...