1. Installing JOREK on Eurofusion gateway.
Part of the instructions is copied from JOREK wiki pages, eventually instructions found here should overlap with those from JOREK wiki.
1.1. How to obtain JOREK source code - ITER git platform
- Request an account for the ITER platform via email.
- Request access to JOREK code
- Create an ssh keypair (help)
Run the command
ssh-keygen -t rsa -C "<key identifier>"
Example:
ssh-keygen -t rsa -C "git.iter.org"
You will be prompted for a location to save your key. Be careful not to overwrite any existing keys that you still need!
You can add a password for an additional layer of security (encrypting your private key file) or proceed without a password. Push Enter when prompted for a password if you don't want to add a password.
Upload your public ssh key: https://git.iter.org/plugins/servlet/ssh/account/keys
The public key is stored in the .pub file.
The .pub file is a regular text file. You will need to copy its contents into the web form that you get after clicking “Add key”.
If you are using other keys, configure a .ssh/config file by adding:
Host git.iter.org HostName git.iter.org IdentityFile ~/.ssh/id_rsa_iter
if you named your file id_rsa_iter,
- Get the code from ITER platform
Your home directory is located on AFS file system (/afs/eufus.eu/user/g/<username>) which is protected by ACL and by default is not accessible by batch. If you intend to submit your jobs to batch system it is better to use GPFS file system. Save your repository for example in /pfs/work/<username>/jorek folder. You can read more about Gateway Storage Areas here.
Clone JOREK repository
git clone ssh://git@git.iter.org/stab/jorek.git
1.2. Compiling JOREK
The following modules need to be loaded e.g. by including in the ~/.bashrc
module purge module load cineca module load intel/pe-xe-2017--binary module load intelmpi/2017--binary module load mkl/2017--binary module load fftw/3.3.4--intelmpi--2017--binary module load szip/2.1--gnu--6.1.0 module load zlib/1.2.8--gnu--6.1.0 module load hdf5/1.8.17--intelmpi--2017--binary
Before compilation one has to specify which compiler and libraries will be used. This depends on the machine. For the Eurofusion Gateway the following Makefile.inc for compiling JOREK needs to be copied to JOREK repository directory
The above Makefile.inc can be modified. In line 2 the model is defined. There are several models available 199, 303, 401, 500, 501, 502, 600, 710, 711 and 712. The list of input parameters for each model can be found here. For the demonstration purposes model 303 is chosen.
Most of the necessary libraries are already available at the Gateway. There are two libraries Pastix (5.2.3) and Scotch (5.1.12) which are not available and have to be provided separately. The precompiled libraries can be found in ~g2iholod/public folder. The apropriate path is set in line 23 of Makefile.inc. Depending on the setting MUMPS library may also be needed but it is not used in the example run (see line19 of Makefile.inc).Compile JOREK with
make -j 8
This creates an executable file "jorek_model<model_number>" which is "jorek_model303" in our case.
1.3. Submitting jobs
Example of the batch job submission script
#!/bin/bash #SBATCH --job-name=jorek #SBATCH --nodes=1 #SBATCH --ntasks=2 #SBATCH --cpus-per-task=24 #SBATCH --time=00:10:00 #SBATCH --partition=gw export OMP_NUM_THREADS=24 srun /pfs/work/$USER/jorek/jorek_model303 < ./inxflow > logfile.out
The "inxflow" file with example input parameters can be found in ~g2iholod/public/jorek_dev/tmp directory. Since the example run is expected to finished within 30 minutes it is possible to change the standard partition "gw" to the debugging partition "gwdbg" by setting
#SBATCH --partition=gwdbg
in submit.sh script and omit the queue for jobs that take more time.
It is advised to run the script in a different folder than the repository. Output files are saved in a working directory and running jorek in the repository results in mixing source files with input and output files.
Submit the bash script with the command
sbatch submit.sh
Check your submission status with
squeue -u $USER
1.4. Creating namelist
Procedure of creating a JOREK namelist for a given model number is a variation of running a "JOREK for the first time case" instruction described in jorek wiki. As the access to JOREK wiki is restricted we describe the procedure step by step here.
It is reccomended that you have access to JOREK repository. This way, you are sure that you are using the latest version of the code and namelist. Furthermore, it allows you to test for different models easily yourself (the namelists of the models are not exactly identical).
Take the very simple intear test case from the tutorial and run it for model199 with 0 time steps (nstep=0):
===== - Running a simple test case =====
We will run a very simple test case: a **tearing mode in a large aspect-ratio circular plasma**. You will have to find your own way of how to structure directories for your simulations. For the prupose of this instruction we assume you have created the following directory structure with the source code and the ''Makefile.inc'' prepared according to the instructions section in 1.2:
jorek-namelist/ jorek.git/ # the source code intear_ntor3/ # run directory
If you want to clone the repository to a specific folder, 'jorek.git' in our example, you can do it by adding 'folder.git' to 'git clone ...' command:
cd jorek-namelist/ git clone ssh://git@git.iter.org/stab/jorek.git jorek.git
To write out the namelist, you need to add right before the endif in line 193 of in models/model199/initialise_parameters.f90 the following three lines:
open(42, file='out.nml', status='replace', action='write')
write(42,in1)
close(42)
Same for the other models although the line number may change.
First, we compile the code with appropriate settings for the test case:
cd jorek.git/ ./util/config.sh model=199 n_tor=3 n_period=1 n_plane=4 # run with toroidal mode numbers n=0,1 make clean make -j 8 make -j 8 jorek2vtk jorek2_postproc
Then, we copy the compiled binaries and the input file for the test case into the run directory (the input file ''intear'' contains the information necessary to run our test case, more information about the namelist input files is given further below):
cp jorek_model199 jorek2vtk jorek2_postproc namelist/model199/intear ../intear_ntor3 cd ../intear_ntor3
Now, open with your favorite editor the file ''intear'', which contains the description of the test case to be simulated. Many things here will not immediately be self-explanatory of course. The only thing we do now is to change the settings for the time stepping by setting
tstep = 3000. nstep = 0
Important notice: You can run the other models using the same intear input file as well. However, it is not guaranteed that it works for all the models.