...
Code Block |
---|
|
export WORK=/pfs/work/$USER
export SCRATCH=/pfs/scratch/$USER mkdir mkdir -p ~/.local/bin ${SCRATCH}
ln -s ${WORK}/dot-udocker/udocker ~/.udocker
wget https://github.com/indigo-dc/udocker/releases/download/1.3.4/udocker-1.3.4.tar.gz
tar zxvf udocker-1.3.4.tar.gz -C ${WORK}/
chmod u+rx ${WORK}/udocker
echo "tmpdir='${SCRATCH}'" > ~/.udocker/udocker.conf
#test udocker exec
${WORK}/udocker/udocker --help
#install
${WORK}/udocker/udocker install |
...
Info |
---|
title | uDocker on other machines |
---|
|
The same steps can be repeated on other machines, including HPC or your local computer. Just change WORK and SCRATCH variables in the first two lines. Note, that SCRATCH will be used during import of new images and WORK will be used to store images and files on all running containers. Choose them wisely to accommodate possibly big amount of data. |
...
...
...
|
If ${WORK}/udocker/udocker command throws errors try to add python3 command before path. Code Block |
---|
module load itm-python/3.10
python3 ${WORK}/udocker/udocker --help
python3 ${WORK}/udocker/udocker install |
|
Exercises
Start uDocker container
- Download latest Ubuntu image:
${WORK}/udocker
Download latest Ubuntu image: ~/.local/bin/udocker pull ubuntu
- Verify available images:
~/.local/bin/udocker images
${WORK}/udocker/udocker
images
- Create a container:
~/.local/bin${WORK}/udocker/udocker
create create --name=udocker-ubuntu ubuntu
- List running containers:
~/.local/bin${WORK}/udocker/udocker ps
ps
- Run an interactive shell:
~/.local/bin/udocker run ${WORK}/udocker/udocker
run udocker-ubuntu /bin/bash
- Check user:
whoami
- Check operating system:
cat /etc/os-release
- Exit from the container:
exit
- Check operating system at Gateway:
cat /etc/os-release
- Delete the container:
~/.local/bin${WORK}/udocker/udocker rm
rm udocker-ubuntu
Info |
---|
As in case of Singularity , here, you can spot the difference as well. As long as you are inside uDocker based container you will see Code Block |
---|
> cat /etc/os-release
NAME="Ubuntu"
VERSION="18.04.3 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
...
... |
while Gateway node reports itself as Code Block |
---|
> cat /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
...
... |
|
...
To load the images, please run:
Code Block |
---|
~/.local/bin/${WORK}/udocker/udocker load -i ~g2tomz/public/imas-fc2k-latest.tar.xz
~/.local/bin${WORK}/udocker/udocker load -i ~g2tomz/public/imas-gui-latest.tar.xz |
...
- Make sure that the images are loaded:
~/.local/bin${WORK}/udocker/udocker images
images
- Create container:
~/.local/bin${WORK}/udocker/udocker
create create --name=imas imas/fc2k:3.24.0-4.2.0-2.5p4-3.0.5-4.6.5
- Run an interactive shell:
~/.local/bin${WORK}/udocker/udocker run
run imas /bin/bash
Prepare and run a Python script which will create IDS pf_active in shot 1 and run 1:
Code Block |
---|
cat << EOF > put_pf.py
import imas
if __name__ == '__main__':
ids = imas.ids(1, 1, 1, 1)
ids.create_env('imas', 'test', '3')
ids.pf_active.ids_properties.comment = 'Test data'
ids.pf_active.ids_properties.homogeneous_time = 0
ids.pf_active.coil.resize(2)
ids.pf_active.coil[0].name = 'COIL 1A'
ids.pf_active.coil[1].name = 'COIL 2B'
number = 10
ids.pf_active.coil[0].current.data.resize(number)
ids.pf_active.coil[0].current.time.resize(number)
for i in range(number):
ids.pf_active.coil[0].current.data[i] = 2 * i
ids.pf_active.coil[0].current.time[i] = i
number = number + 2
ids.pf_active.coil[1].current.data.resize(number)
ids.pf_active.coil[1].current.time.resize(number)
for i in range(number):
ids.pf_active.coil[1].current.data[i] = 2 * i + 10
ids.pf_active.coil[1].current.time[i] = i + number
ids.pf_active.put()
EOF
python put_pf.py |
- You can now exit container shell. Note, that the container is not executing anything, but it persists and all files are kept in
~/.udocker
. Copy the generated pulsefile directly into your Gateway's collection of pulsefiles:
cp ~${WORK}/.udocker/containers/imas/ROOT/home/imas/public/imasdb/test/3/0/ids_10001.* ~/public/imasdb/test/3/0/
- Verify that the IDS pf_active has been created:
idsdump $USER test 3 1 1 pf_active
Running Kepler workflow in a container
- Create container:
~/.local/bin/${WORK}/udocker/
udocker create --name=imas-gui imas/gui:3.24.0-4.2.0-2.5p4-3.0.5-4.6.5
- Run the default application (VNC server) with port mapping:
~/.local/bin${WORK}/udocker/udocker
run run --publish 15901:5901 imas-gui
- In another terminal, open VNC viewer, connect to
localhost:15901
and use imas
as the password: vncviewer localhost:15901
- You will see Openbox desktop environment, with Kepler loading automatically (please wait until it is ready).
- Design an example workflow like the one below, which will read pf_active IDS from shot 1 and run 1:
![](/download/attachments/24088097/2019-10-25-133430_459x308_scrot.png?version=1&modificationDate=1572003393613&api=v2)
- Run the workflow and notice that it fails due to lack of pulsefile with required content. This is because in the previous exercise you were running in the imas container, isolated from the imas-gui one.
- You can now do one of the following:
- Create the pulsefile again
- Run in another terminal in the Gateway:
~/.local/bin/udocker run ${WORK}/udocker/udocker
run imas-gui /bin/bash
- Repeat step 4 from previous exercise
- Copy pulsefile between containers
- Run
cp ~${WORK}/.udocker/containers/imas/ROOT/home/imas/public/imasdb/test/3/0/ids_10001.* ~${WORK}/.udocker/containers/imas-gui/ROOT/home/imas/public/imasdb/test/3/0/
- Start the workflow again. The expected result:
![](/download/attachments/24088097/2019-10-25-133452_604x462_scrot.png?version=1&modificationDate=1572003443067&api=v2)
...