Info | ||
---|---|---|
| ||
ANAKIN: Is it possible to learn this power? Star Wars: Revenge of the Sith |
This tutorial is about working with IMAS in an isolated environment (containers), which you can instantiate anytime, copy and reuse with little effort on different machines. After this tutorial you will gain knowledge of:
...
- To download an image from Docker Hub, use:
docker pull <image-name>
- To list available images, use:
docker images
- To remove image, use:
docker rmi <image-name>
Example 1. Pulling an image
Containers
- To run a container, use:
docker run <image-name> <command>
- To list containers, use:
docker ps
- To copy a file from/into a running container, use:
docker cp <file-1> <file-2>
- To execute a command in a running container, use:
docker exec <container-id> <command>
- To remove container, use:
docker rm <container-id>
Info | ||
---|---|---|
| ||
Each command supports additional flags passed along the main arguments. Please make sure to check The most often used flags are:
|
Examples
Pulling an image
Example 2. Starting a container
Exercises
Checking /etc/os-release of different containers
...
uDocker is contained in a single Python file making it easy to deploy in distributed systems. To install it on Gateway execute:
Code Block | ||
---|---|---|
| ||
setenvexport WORK =/pfs/work/$USER setenv SCRATCH /pfs/scratch/$USER mkdir -p ~/.local/bin ${WORK}/dot-udocker ${SCRATCH} ln -s ${WORK}/dot-udocker ~/.udocker curl wget https://rawgithub.githubusercontent.com/indigo-dc/udocker/releases/masterdownload/udocker.py > ~/.local/bin/udocker chmod u+rx ~/.local/bin/udocker echo "tmpdir='${SCRATCH}'" > ~/.udocker/udocker.conf ~/.local/bin/1.3.4/udocker-1.3.4.tar.gz tar zxvf udocker-1.3.4.tar.gz -C ${WORK}/ chmod u+rx ${WORK}/udocker #test udocker exec ${WORK}/udocker/udocker --help #install ${WORK}/udocker/udocker install |
Info | ||
---|---|---|
| ||
The same steps can be repeated on other machines, including HPC or your local computer. Just change WORK and SCRATCH variables in the first two lines. Note, that SCRATCH will be used during import of new images and WORK will be used to store images and files on all running containers. Choose them wisely to accommodate possibly big amount of data. |
...
Info | |
---|---|
|
...
|
...
| ||
If
|
Exercises
Start uDocker container
- Download latest Ubuntu image:
${WORK}/udocker/udocker pull ubuntu
- Verify available images:
${WORK}/udocker/udocker
images
- Create a container:
${WORK}/udocker/udocker
create --name=udocker-ubuntu ubuntu
- List running containers:
${WORK}/udocker/udocker
ps
- Run an interactive shell:
${WORK}/udocker/udocker
run udocker-ubuntu /bin/bash
- Check user:
whoami
- Check operating system:
cat /etc/os-release
- Exit from the container:
exit
- Check operating system at Gateway:
cat /etc/os-release
- Delete the container:
${WORK}/udocker/udocker
rm udocker-ubuntu
Info |
---|
As in case of |
- Download latest Ubuntu image:
~/.local/bin/udocker pull ubuntu
- Verify available images:
~/.local/bin/udocker images
- Create a container:
~/.local/bin/udocker create --name=udocker-ubuntu ubuntu
- List running containers:
~/.local/bin/udocker ps
- Run an interactive shell:
~/.local/bin/udocker run udocker-ubuntu /bin/bash
- Check user:
whoami
- Check operating system:
cat /etc/os-release
- Exit from the container:
exit
- Check operating system at Gateway:
cat /etc/os-release
- Delete the container:
~/.local/bin/udocker rm udocker-ubuntu
Info | ||||
---|---|---|---|---|
As in case of
while
|
...
To load the images, please run:
Code Block |
---|
~/.local/bin${WORK}/udocker/udocker load -i ~g2tomz/public/imas-fc2k-latest.tar.xz ~/.local/bin/${WORK}/udocker/udocker load -i ~g2tomz/public/imas-gui-latest.tar.xz |
...
- Make sure that the images are loaded:
~/.local/bin${WORK}/udocker/udocker images
images
- Create container:
~/.local/bin${WORK}/udocker/udocker
create create --name=imas imas/fc2k:3.24.0-4.2.0-2.5p4-3.0.5-4.6.5
- Run an interactive shell:
~/.local/bin${WORK}/udocker/udocker run
run imas /bin/bash
Prepare and run a Python script which will create IDS pf_active in shot 1 and run 1:
Code Block cat << EOF > put_pf.py import imas if __name__ == '__main__': ids = imas.ids(1, 1, 1, 1) ids.create_env('imas', 'test', '3') ids.pf_active.ids_properties.comment = 'Test data' ids.pf_active.ids_properties.homogeneous_time = 0 ids.pf_active.coil.resize(2) ids.pf_active.coil[0].name = 'COIL 1A' ids.pf_active.coil[1].name = 'COIL 2B' number = 10 ids.pf_active.coil[0].current.data.resize(number) ids.pf_active.coil[0].current.time.resize(number) for i in range(number): ids.pf_active.coil[0].current.data[i] = 2 * i ids.pf_active.coil[0].current.time[i] = i number = number + 2 ids.pf_active.coil[1].current.data.resize(number) ids.pf_active.coil[1].current.time.resize(number) for i in range(number): ids.pf_active.coil[1].current.data[i] = 2 * i + 10 ids.pf_active.coil[1].current.time[i] = i + number ids.pf_active.put() EOF python put_pf.py
- You can now exit container shell. Note, that the container is not executing anything, but it persists and all files are kept in
~/.udocker
. Copy the generated pulsefile directly into your Gateway's collection of pulsefiles:cp ~/.${WORK}/udocker/containers/imas/ROOT/home/imas/public/imasdb/test/3/0/ids_10001.* ~/public/imasdb/test/3/0/
- Verify that the IDS pf_active has been created:
idsdump $USER test 3 1 1 pf_active
Running Kepler workflow in a container
- Create container:
~/.local/bin/${WORK}/udocker/
udocker create --name=imas-gui imas/gui:3.24.0-4.2.0-2.5p4-3.0.5-4.6.5
- Run the default application (VNC server) with port mapping:
~/.local/bin${WORK}/udocker/udocker
run run --publish 15901:5901 imas-gui
- In another terminal, open VNC viewer, connect to
localhost:15901
and useimas
as the password:vncviewer localhost:15901
- You will see XFCE4 Openbox desktop environment. Open terminal and run:
kepler
, with Kepler loading automatically (please wait until it is ready). - Design an example workflow like the one below (it reads , which will read pf_active IDS from shot 1 and run 1 as created by previous step):
: - Run the workflow and notice that it fails due to lack of pulsefile with required content. This is because in the previous exercise you were running in the imas container, isolated from the imas-gui one.
- You can now do one of the following:
- Create the pulsefile again
- Run in another terminal in the Gateway:
${WORK}/udocker/udocker
run imas-gui /bin/bash
- Repeat step 4 from previous exercise
- Run in another terminal in the Gateway:
- Copy pulsefile between containers
- Run
cp ${WORK}/udocker/containers/imas/ROOT/home/imas/public/imasdb/test/3/0/ids_10001.* ${WORK}/udocker/containers/imas-gui/ROOT/home/imas/public/imasdb/test/3/0/
- Run
- Create the pulsefile again
- Start the workflow again. The expected result: