...
Install SUMI on the Gateway
Configure sample job and big Marconi connection
Test connection with Marconi
Install uDocker on Marconi
Test image on Marconi
- Configure sample job
Submit workflow
Retrieve data
Visualize output on GW
...
To configure a job we have to edit the files copied on the
jobs.conf
The configuration file jobs.conf located at local directory $HOME/.sumi/ contains the configuration for the jobs to be run. The sample configuration file located at $SUMI_DIR/conf/jobs.conf has the following content.
[test]
udocker = udocker.py
arguments =
cpus = 1
time = 1
threads_per_process = 1
servers.conf
The configuration file servers.conf located at local directory $HOME/.sumi/ contains the configuration for the servers where SUMI will connect The sample configuration file located at $SUMI_DIR/conf/servers.conf has the following content.
[machine]
server = example.com
user = username
manager = slurm
protocol = ssh
To configure the login node of y our cluster just specify the login node address, your user name and the name of the resource manager where the accepted are sge, slurm and pbs.
Test connection with Marconi
To use SUMI we need a passwordless connection. This means that Marconi needs to have the public key of our account at Gateway on the list of allowed keys. To copy our key we make usage of the "ssh-copy-id" command
ssh-copy-id username@login.marconi.cineca.it
Now, if we stablish an SSH connection, the prompt will not ask for any password and will give us a terminal inside Marconi.
ssh username@login.marconi.cineca.it
Installing uDocker
We will make usage of uDocker. uDocker is based on Docker but it does not require root permissions. Therefore, it does not have some features as TCP/IP sniffing or other action that require root privileges. uDocker will allow to simulate an environment with root permissions (UID = 0) and install IMAS inside.
Therefore, the very first step is installing uDocker. uDokcer will be set up once and used many times
...
Once the files have been copied, edit the setup_udocker.sh script. Set up the variables IMAS_IMAGE, UDOCKER_TARBALL and then run the script.
bash setup_udocker.sh
This script will install uDocker as well as create the IMAS container inside uDocker.
udocker but it will not load the Docker image inside. For this, we need to run the following commands
$SUMI_UDOCKER_DIR/udocker.py load -i $IMAS_IMAGE
$SUMI_UDOCKER_DIR/udocker.py create --name=imas imas-installer:20180529181447
Known issues
When setting up a uDocker image locally be sure that there is enough quota for your user. Otherwise it may crash or show untar related problems as the following when running the udocker load command.
Error: failed to extract container:
Error: loading failed
...
SAGA and Paramiko require a passwordless connection to work.
Copy your public key to the remote host
ssh-copy-id -i ~/.ssh/id_rsa.pub username@example.com
jobs.conf
...
servers.conf
The configuration file servers.conf located at local directory $HOME/.sumi/ contains the configuration for the servers where SUMI will connect The sample configuration file located at $SUMI_DIR/conf/servers.conf has the following content.
[machine]
server = example.com
user = username
manager = slurm
protocol = ssh
To configure the login node of y our cluster just specify the login node address, your user name and the name of the resource manager where the accepted are sge, slurm and pbs.
Remote execution
There can be different machines and different jobs specified. Every time that we run SUMI we have to specify to which supercomputer are we submitting to and which of the configured jobs are we going to submit. Otherwise SUMI does not run anything and is conservative. To specify the job and machine we have to use the parameters -j and -m respectively. For example to run a given job named testjob and a machine testmachine we should execute the following command:
sumi -r -j testjob -m testmachine
Be aware that SUMI is case sensitive when specifying the names.
...