...
Inside the "wf" directory we will also have a new version of our "simple-workflow.xml" file which will overwrite the existing one. These files will be copied inside the image which is contained in the directory ".udocker/containers/imas/ROOT/". Once our job has finished we want to copy the IDS files to our local GatewaGateway/ITER. The following configuration is for a Gateway machine, but this only affect to the paths of "upload_files" and "download_to"
| Code Block |
|---|
[marconi] server = login.marconi.cineca.it user = USER manager = slurm protocol = ssh upload_files = /afs/eufus.eu/g2itmdev/USER/mywf/* upload_to = /marconi/home/userexternal/USER/.udocker/containers/imas/ROOT/home/imas/ download_files = /marconi/home/userexternal/USER/.udocker/containers/imas/ROOT/home/imas/public/imasdb/test/3/0/* download_to = /afs/eufus.eu/g2itmdev/user/USER/public/imasdb/test/3/0/ |
asdThe job configuration is the following
| Code Block |
|---|
[test] udocker = $HOME/.local/bin/udocker arguments = run imas /bin/bash -l script.sh cpus = 1 time = 20 threads_per_process = 1 |
...
| Code Block |
|---|
sumi -r -j test -m marconi |
The output will
2018/12/19 17:09:47 INFO SUMI: Starting
2018/12/19 17:09:47 INFO SUMI: Reading local configuration
2018/12/19 17:09:47 INFO Job: configuring
2018/12/19 17:09:52 INFO SUMI: uploading files
2018/12/19 17:09:52 INFO Connected (version 2.0, client OpenSSH_6.6.1)
2018/12/19 17:09:53 INFO Authentication (publickey) successful!
Copy the files from Gatr
2018/12/19 17:09:54 INFO [chan 0] Opened sftp connection (server version 3)
2018/12/19 17:09:54 INFO SUMI: scp /afs/eufus.eu/g2itmdev/user/USER/mywf/simple-workflow.xml /marconi/home/userexternal/USER/.udocker/containers/imas/ROOT/home/imas/
2018/12/19 17:09:54 INFO SUMI: scp /afs/eufus.eu/g2itmdev/user/USER/mywf/script.sh /marconi/home/userexternal/USER/.udocker/containers/imas/ROOT/home/imas/
2018/12/19 17:09:54 INFO [chan 0] sftp session closed.
2018/12/19 17:09:59 INFO Job: starting
2018/12/19 17:09:59 INFO Job: ID [slurm+ssh://login.marconi.cineca.it]-[3264973]
2018/12/19 17:10:03 INFO Job: state Pending
2018/12/19 17:10:03 INFO Job: waiting
2018/12/19 17:17:13 INFO Job: State Done
2018/12/19 17:17:13 INFO Job: Exitcode 0
2018/12/19 17:17:13 INFO SUMI: downloading files
2018/12/19 17:17:13 INFO Connected (version 2.0, client OpenSSH_6.6.1)
2018/12/19 17:17:14 INFO Authentication (publickey) successful!
2018/12/19 17:17:14 INFO [chan 1] Opened sftp connection (server version 3)
2018/12/19 17:17:14 INFO SUMI: scp /marconi/home/userexternal/USER/.udocker/containers/imas/ROOT/home/imas/public/imasdb/test/3/0/ids_10001.characteristics /afs/eufus.eu/g2itmdev/user/user/public/imasdb/test/3/0/
2018/12/19 17:17:23 INFO SUMI: scp /marconi/home/userexternal/USER/.udocker/containers/imas/ROOT/home/imas/public/imasdb/test/3/0/ids_10001.datafile /afs/eufus.eu/g2itmdev/user/user/public/imasdb/test/3/0/
2018/12/19 17:17:23 INFO SUMI: scp /marconi/home/userexternal/USER/.udocker/containers/imas/ROOT/home/imas/public/imasdb/test/3/0/ids_10001.tree /afs/eufus.eu/g2itmdev/user/user/public/imasdb/test/3/0/
2018/12/19 17:17:31 INFO SUMI: scp /marconi/home/userexternal/USER/.udocker/containers/imas/ROOT/home/imas/public/imasdb/test/3/0/ids_19999.characteristics /afs/eufus.eu/g2itmdev/user/user/public/imasdb/test/3/0/
2018/12/19 17:17:39 INFO SUMI: scp /marconi/home/userexternal/USER/.udocker/containers/imas/ROOT/home/imas/public/imasdb/test/3/0/ids_19999.datafile /afs/eufus.eu/g2itmdev/user/user/public/imasdb/test/3/0/
2018/12/19 17:17:39 INFO SUMI: scp /marconi/home/userexternal/USER/.udocker/containers/imas/ROOT/home/imas/public/imasdb/test/3/0/ids_19999.tree /afs/eufus.eu/g2itmdev/user/user/public/imasdb/test/3/0/
2018/12/19 17:17:47 INFO [chan 1] sftp session closed.
2018/12/19 17:17:47 INFO SUMI: Done
This will copy the files, run the workflow and retrieve the output results. Once we have them we can check whether the
...