1. 1.How to set environment to use IMAS FC2K Testing Framework

1.1. 1.1.Prepare basic IMAS environment

$> setenv KEPLER_DIR <keplers dir e.g. $ITMWORK/imas-keplers>
$> module load imasenv
IMAS environment loaded.

1.2. 1.2. Install or load Kepler

$> kepler_install <version>
OR
$> kepler_load <version>

1.3. 1.3. Prepare environment for IMAS FC2K Testing Framework

$> module load imas-pyal
$> imasdb test


2. 2. Checking out IMAS FC2K Testing Framework

IMAS FC2K Testing Framework is a set of components that allow to test whole chain of execution. From native code compilation to workflow execution.

$> git clone https://{ITER_USERNAME}@git.iter.org/scm/imex/fc2k-testing-framework.git
$> cd fc2k-testing-framework
$> ./run_test.sh
--- Starting test for: eq2eq ---
Actor: eq2eq - OK
Skipping log removal
--- Finished test for: eq2eq ---
--- Starting test for: eq2eqmpi ---
Actor: eq2eqmpi - OK
Skipping log removal
--- Finished test for: eq2eqmpi ---
--- Starting test for: mpi ---
Actor: mpi - OK
Skipping log removal
--- Finished test for: mpi ---
--- Starting test for: nocpo ---
Actor: nocpo - OK
Skipping log removal
--- Finished test for: nocpo ---
--- Starting test for: nocpo_param ---
Actor: nocpo_param - OK
Skipping log removal
--- Finished test for: nocpo_param ---
--- Starting test for: stringinout ---
Actor: stringinout - OK
Skipping log removal
--- Finished test for: stringinout ---

then a test report will appear inside each directory:

./eq2eqmpi/test.log
./mpi/test.log
./eq2eq/test.log
./stringinout/test.log
./nocpo/test.log
./nocpo_param/test.log

3. 3. Integrating new actor with Testing Framework

3.1. 3.1. Minimal Test Case

It is possible to integrate new actor with FC2K Testing Framework, however, it requires following a convention. Basically, the most simple test can provide only one script

run_test.sh

This test is responsible for running and reporting status of test execution.

3.2. 3.2. Set of tests provided by CPT

Default set of tests, provides testing of common cases based on FC2K and Kepler. There are few steps that each tests performs. If you want to make your own tests, I suggest to perform the same steps.

- create input data
   - either create input pulse file
   - or copy it from location
- compile native code (actor)
- create actor - based on your own FC2K project
   - when preparing FC2K project, make sure to use relative paths (e.g.: "./libactor.so")
- run workflow - you need your own workflow that runs actor
   - if you don't want to use Kepler, you need to provide another means of running native code (e.g.: Python based wrapper)
- compare results with expected ones
   - you have to provide means of determining whether execution was correct or not
   - you need to provide data set (expected) that will be compared with actual output
   - you should report whether test passed or failed
- you should clean up after execution 

3.3. 3.3. Example of Test Case - based on MPI

Let's take a look at one of Test Cases provided by FC2K Testing Framework. It consist of few files that serve given purpose.

These files are:

  • FC2K projects
  • Kepler workflows
  • expected output
  • Make file
  • execution script

mpi
|-- expected_output.txt                - this file contains expected output of execution
|-- fc2k_nocpo_mpi_batch.xml           - actor that will be started on batch nodes
|-- fc2k_nocpo_mpi_host.xml            - actor that will be started on Gateway machine
|-- fc2k_nocpo_mpi_open_batch.xml      - actor that will use OpenMP in batch mode
|-- fc2k_nocpo_mpi_open_host.xml       - actor that will use OpenMP on Gateway
|-- Makefile                           - provides recipe for building actor's library
|-- nocpompi.f90                       - source code of actor
|-- run_test.sh                        - executes tests and reports the outcome
`-- workflow_mpi_chain.xml             - workflow that runs all the actors

3.4. 3.4. Developing simple execution script

Below, you can find minimal test case with all the steps required to run it

#!/bin/bash
 
# ../common.sh contains helper functions
source ../common.sh
 
# init is a wrapper for calling make (building actor library)
init_test

# we are building all the actors here
${FC2K}/bin/fc2k fc2k_nocpo_mpi_batch.xml >> test.log 2>&1
${FC2K}/bin/fc2k fc2k_nocpo_mpi_host.xml >> test.log 2>&1
${FC2K}/bin/fc2k fc2k_nocpo_mpi_open_batch.xml >> test.log 2>&1
${FC2K}/bin/fc2k fc2k_nocpo_mpi_open_host.xml >> test.log 2>&1
 
# start_kepler is a helper function defined in ../common.sh
# it starts workflow
start_kepler workflow_mpi_chain.xml
 
# this is the place where checking results occurs. Note that it is up to developer
# to define what does it mean to pass/fail the test
diff redirect/workflow_mpi_chain.Display.txt expected_output.txt >> test.log 2>&1
 
# status_code is a helper function that shows final result of test
# based on error code from previous call
status_code mpi false

3.5. 3.5. Common functions

FC2K Testing Framework provides set of helper functions that make it simpler to create run_test.sh script.

status_code   - this function takes two arguments: name of the test and boolean argument
                - name is used to show the name of the test next to execution status
                - boolean flag is used to determine whether logs should be removed or not
                - this function uses lats exit code ($?) to determine whether test passed or failed
 
init_test     - this function takes no arguments. It simply runs: 
                  make clean
                  make
                  recreates directory "redirect"
 
start_kepler  - takes one parameter - workflow name
                  starts workflow

Acknowledgement

This work has been carried out within the framework of the EUROfusion Consortium and has received funding from the Euratom research and training programme 2014-2018 under grant agreement No 633053.The scientific work is published for the realization of the international project co-financed by Polish Ministry of Science and Higher Education in 2019 and 2020 from financial resources of the program entitled "PMW"; Agreement No. 5040/H2020/Euratom/2019/2 and 5142/H2020-Euratom/2020/2”.

  • No labels