Framework directory structure
Your repository:
Assuming your repository looks something like this:
Code Block |
---|
|
Project_root
├── root_files.extension
├── root_directiories/
│ ├── directories/
│ ├── ...
│ └── files.extension
└── tests # <- this is directory where the whole actor testing framework should be placed |
Then the actor testing framework has to be in a directory called tests/
and this is mandatory.
Actor Testing Framework repository - inside your project:
The inside of the tests/
directory, inside your repository, should contain the entire actor testing framework repository like structure below:
Code Block |
---|
|
tests
├── __init__.py
├── ci_scripts
│ ├── .gitlab-ci.yml
│ └── .iter-ci.sh
├── configs
│ ├── actor_configuration.yml
│ └── ids_configuration.yml
├── input_pulse_file
│ ├── ITER
│ │ └── 3
│ │ └── 0
│ │ └── {pulsefiles}
│ └── GITLAB
│ └── 3
│ └── 0
│ └── {pulsefiles}
├── main_code_run
│ └── run_actor.sh
├── modules
│ ├── modules_gitlab.yml
│ └── modules_iter.yml
├── native_codes_build
│ └── build_actor.sh
├── pytest_scripts
│ ├── conftest.py
│ ├── __init__.py
│ ├── pytest.ini
│ └── test_actor.py
├── README.md
├── test.sh
└── utils
├── get_actor_info.py
├── get_ids_info.py
└── get_ids_name.py |
CI engine - ITER or GITLAB
Actor Testing Framework is prepared to run on both ITER CI (called Bamboo) and GITLAB CI (works like Gateway).
There is a slight difference between these platforms that causes the framework to be configured differently to run tests on each of them.
ITER CI Configuration
ITER Modules
To set up the ITER modules needed to build and run the actor, there is a file called modules/modules_iter.yml
, where you need to provide all the necessary modules, preferably full versions.
Code Block |
---|
|
├── modules
│ ├── modules_gitlab.yml
│ └── modules_iter.yml # <-- This is file dedicated for ITER modules
##############################################
- example: inside modules_iter.yml:
##############################################
modules:
- IMAS
- iWrap
- INTERPOS/9.1.0-intel-2020b
- XMLlib/3.3.1-intel-2020b |
Configuration files
Actor configuration
There is YAML file in the configs directory called actor_configuration.yml
that contain metadata about the actor being tested. For now, only the fields: actor name
and relative path
from the project root to the actor description file are to be set.
Code Block |
---|
|
├── configs
│ ├── actor_configuration.yml # <-- this file
│ └── ids_configuration.yml
##############################################
- example: inside actor_configuration.yml:
##############################################
actor_name: NAME
actor_path: actor/description/file.yaml |
IDS configuration
The Actor Testing Framework assumes that the user needs to prepare input files, output files, and expected (reference) IDS pulse files to be used in the execution of the workflow and tests. On the user side, these files can be placed in the public, user, or custom imasdb path, but only the public and custom imasdb path can be used for the remote CI. This is set in the ids_configuration.yml
file and must match the pulse files and IDS used by the actor workflow.
Code Block |
---|
|
├── configs
│ ├── actor_configuration.yml
│ └── ids_configuration.yml # <-- this file
##############################################
- example: inside ids_configuration.yml:
##############################################
input_ids:
user: ./tests/input_pulse_file
output_ids:
database: ITER
user: ./tests/input_pulse_file/
shot: 114100
run: 42
expected_ids:
database: ITER
user: ./tests/input_pulse_file/
shot: 114100
run: 43
|
For your convenience, the Actor Testing Framework comes with a custom imasdb, located under: testing/input_pulse_file/ITER
directory path that must be accessed from the relative path of the project root directory.
CI script
Repository side CI scripts
ITER has a convention to put all CI-related scripts in the ci_scripts/
directory. Actor Testing Framework follows this convention, but can be extended with an additional file in the root directory: ci_scripts/ci_atf.sh
. Then, in this upper CI script, you need to execute the lower file: tests/ci_scripts/.iter-ci.sh
.
Code Block |
---|
|
Project_root
├── ci_scripts/
│ ├── ci_atf.sh # <-- Upper CI script
│ ├── ...
│ └── ci_...sh
│
├── root_files.extension
├── root_directiories/
│
└── tests/
└── ci_scripts/
└── .iter-ci.sh # <-- Lower CI script
##############################################
- example: inside upper ci script: ci_atf.sh
##############################################
#!/bin/sh
./$(dirname $0)/../tests/ci_scripts/.iter-ci.sh |
Remote ITER CI side
On the remote side of ITER CI - Bamboo CI, you need to create a dedicated job for the Actor Testing Framework and a task to run the CI script. The image below shows how such a task can be configured with the top CI script.
Image Modified
Building Actor Native Codes
Inside the Actor Testing Framework test directory there is a dedicated directory for building native actor code instructions called: native_codes_build/
. By default, there is a build_actor.sh
script that should provide step-by-step instructions on how to build an actor. For example, it could be a simple make
command, but first you need to change the directory your codes are in, basically it should be done the same way you build the codes from the root directory.
Code Block |
---|
|
tests
├── native_codes_build
│ └── build_actor.sh # <-- This file
.
.
. |
Run Actor
Inside the Actor Testing Framework test directory there is a dedicated directory to run the actor code, usually it will be a workflow, called main_code_run/
. By default, there is a run_actor.sh
script that should provide step-by-step instructions for starting an actor or executing a workflow. Basically, it should reflect the same way you run actor from the root directory.
Code Block |
---|
|
tests
├── main_code_run
│ └── run_actor.sh # <-- This file
.
.
. |
Test Cases - Pytest
Actor Testing Framework runs tests using the Python Pytest framework. This requires the user to have a basic understanding of the Pytest framework. All query related scripts are placed in the tests/pytest_scripts
directory. The user can provide as many test cases as needed and is responsible for writing them.
Code Block |
---|
|
tests
├── pytest_scripts
│ ├── conftest.py
│ ├── __init__.py
│ ├── pytest.ini
│ └── test_actor.py
.
.
. |
GILTAB CI Configuration
GITLAB Modules
To set up the GITLAB modules needed to build and run the actor, there is a file called modules/modules_gitlab.yml
, where you need to provide all the necessary modules, preferably full versions.
Code Block |
---|
|
├── modules
│ ├── modules_gitlab.yml # <-- This is file dedicated for GITLAB modules
│ └── modules_iter.yml
##############################################
- example: inside modules_gitlab.yml:
##############################################
modules:
- cineca
- imasenv/3.35.0/intel/17.0/rc
- iwrap/0.4.1-py3-10 |
Configuration files
Actor configuration
There is YAML file in the configs directory called actor_configuration.yml
that contain metadata about the actor being tested. For now, only the fields: actor name
and relative path
from the project root to the actor description file are to be set.
Code Block |
---|
|
├── configs
│ ├── actor_configuration.yml # <-- this file
│ └── ids_configuration.yml
##############################################
- example: inside actor_configuration.yml:
##############################################
actor_name: NAME
actor_path: actor/description/file.yaml |
IDS configuration
The Actor Testing Framework assumes that the user needs to prepare input files, output files, and expected (reference) IDS pulse files to be used in the execution of the workflow and tests. On the user side, these files can be placed in the public, user, or custom imasdb path, but only the public and custom imasdb path can be used for the remote CI. This is set in the ids_configuration.yml
file and must match the pulse files and IDS used by the actor workflow.
Code Block |
---|
|
├── configs
│ ├── actor_configuration.yml
│ └── ids_configuration.yml # <-- this file
##############################################
- example: inside ids_configuration.yml:
##############################################
input_ids:
user: ./tests/input_pulse_file
output_ids:
database: GITLAB
user: ./tests/input_pulse_file/
shot: 114100
run: 42
expected_ids:
database: GITLAB
user: ./tests/input_pulse_file/
shot: 114100
run: 43
|
For your convenience, the Actor Testing Framework comes with a custom imasdb, located under: testing/input_pulse_file/GITLAB
directory path that must be accessed from the relative path of the project root directory.
CI script
Repository side CI scripts
GITLAB has a convention to put CI script in the root
directory. Actor Testing Framework follows this convention, but can be extended with an additional file in the root directory: .gitlab-ci.yml
. Then, in this upper CI script, you need to execute the lower file: tests/ci_scripts/.gitlab-ci.yml
.
Code Block |
---|
|
Project_root
├── .gitlab-ci.yml # <-- Upper CI script
│
├── root_files.extension
├── root_directiories/
│
└── tests/
└── ci_scripts/
└── .gitlab-ci.yml # <-- Lower CI script
##############################################
- example: inside upper ci script: .gitlab-ci.yml
##############################################
include:
- 'tests/ci_scripts/.gitlab-ci.yml' |
Building Actor Native Codes
Inside the Actor Testing Framework test directory there is a dedicated directory for building native actor code instructions called: native_codes_build/
. By default, there is a build_actor.sh
script that should provide step-by-step instructions on how to build an actor. For example, it could be a simple make
command, but first you need to change the directory your codes are in, basically it should be done the same way you build the codes from the root directory.
Code Block |
---|
|
tests
├── native_codes_build
│ └── build_actor.sh # <-- This file
.
.
. |
Run Actor
Inside the Actor Testing Framework test directory there is a dedicated directory to run the actor code, usually it will be a workflow, called main_code_run/
. By default, there is a run_actor.sh
script that should provide step-by-step instructions for starting an actor or executing a workflow. Basically, it should reflect the same way you run actor from the root directory.
Code Block |
---|
|
tests
├── main_code_run
│ └── run_actor.sh # <-- This file
.
.
. |
Test Cases - Pytest
Actor Testing Framework runs tests using the Python Pytest framework. This requires the user to have a basic understanding of the Pytest framework. All query related scripts are placed in the tests/pytest_scripts
directory. The user can provide as many test cases as needed and is responsible for writing them.
Code Block |
---|
|
tests
├── pytest_scripts
│ ├── conftest.py
│ ├── __init__.py
│ ├── pytest.ini
│ └── test_actor.py
.
.
. |
Example
The Actor Testing Framework includes