...

In this tutorial the local computer used will be Marconi Gateway. Given that SUMI uses 2.7 python version, first we will load the correspondent module

module load python/2.7.12

SUMI depends on two powerfull libraries to perform its tasks. These are Paramiko (data transfer) and SAGA (remote job submission). To install the dependencies we need to download python libraries, but given that we do not have root permissions we will make usage of a virtual environment which will allow to install these libraries locally. For this purpose we will use "virtualenv". Virtualenv It creates a python environment in local folders and allows to create a virtual python environment where we can install libraries locally. To set it up we run the following commands: 

    mkdir sumi-virtualenv

    virtualenv sumi-virtualenv

...

    git clone https://albertbsc@bitbucket.org/albertbsc/sumi.git

...

This will create a local folder named "sumi". To include it in the $PATH environment variable we need to run the software.

...

SUMI requires two configuration job files which contain the information of the jobs to be sumitted and the HPC cluster where we are going to submit. For this we need to create the configuration folder and copy the configuration files jobs.conf and servers.conf from sumi/conf/ directory. 

    mkdir $HOME/.sumi

    cp sumi/conf/*.conf $HOME/.sumi

...

Now, we are ready to run SUMI. Execute the option "-h" to see all the

$ sumi -h

usage: sumi.py [-h] [-r] [-u UP] [-d DOWN] [-m MACHINE [MACHINE ...]]

               [-j JOBS [JOBS ...]]

Submission Manager for IMAS

optional arguments:

  -h, --help            show this help message and exit

  -r, --run             Run the configured job on a specific cluster

  -u UP, --upload UP    Upload local file

  -d DOWN, --download DOWN

                        Download remote file

  -m MACHINE [MACHINE ...], --machine MACHINE [MACHINE ...]

                        List of machines were to submit

  -j JOBS [JOBS ...], --job JOBS [JOBS ...]

                        List of jobs to be submitted

...

Then, SUMI has been installed successfully.

Passwordless configuration with Marconi

SAGA and Paramiko require a passwordless connection to work.

Copy your public key to the remote host

ssh-copy-id username@login.marconi.cineca.it

Configure sample job and big Marconi connection

...

Download the source code from BitBucket

https://bitbucket.org/albertbsc/sumi/src/master/

Export the variable $SUMI_DIR with the path to SUMI and include the bin directory inside the PATH variable


    #!bash
    export SUMI_DIR=/path/to/sumi
    export PATH=$PATH:$SUMI_DIR/bin
    chmod +x $SUMI_DIR/bin/sumi


If you want these variables to persist includes the previous lines in your .bashrc file.

Passwordless configuration

SAGA and Paramiko require a passwordless connection to work.
Copy your public key to the remote host
    ssh-copy-id -i ~/.ssh/id_rsa.pub username@example.com

Remote execution

There can be different machines and different jobs specified. Every time that we run SUMI we have to specify to which supercomputer are we submitting to and which of the configured jobs are we going to submit. Otherwise SUMI does not run anything and is conservative. To specify the job and machine we have to use the parameters -j and -m respectively. For example to run a given job named testjob and a machine testmachine we should execute the following command:

    sumi -r -j testjob -m testmachine

Be aware that SUMI is case sensitive when specifying the names.

...