1. 1.How to set environment to use FC2K Testing Framework
1.1. 1.1. Prepare Kepler
install_kepler.sh [target dir] [local source Kepler] [description] switch_to_kepler [kepler_name]
1.2. 1.2. Prepare FC2K
If you want to work with most recent codes of FC2K (not released yet), you should either get it from trunk or from development branch.
svn checkout http://gforge6.eufus.eu/svn/fc2k cd fc2k/trunk make source ./setup
If you want to work with development branch (it might have new features but, at the same time, it might be little bit behind current developments inside trunk) you can checkout a particular branch of FC2K.
svn checkout http://gforge6.eufus.eu/svn/fc2k cd fc2k/branches/new_feature make source ./setup
1.3. 1.3. Using installed FC2K release
In case you want to test your actor, and you are not interested in most recent developments inside FC2K, you can always use already installed version of FC2K
> module avail fc2k --------------- /gw/modules/itm -------------- fc2k/R3.0.11 fc2k/R3.0.14 fc2k/R3.0.5 fc2k/rc > module switch fc2k/R3.0.14
In this case, you will run FC2K Testing Framework with a released version of FC2K.
2. 2. Checking out FC2K Testing Framework
FC2K Testing Framework is a set of components that allow to test whole chain of execution. From native code compilation to workflow execution.
$> svn co http://gforge6.eufus.eu/svn/fc2ktf $> cd fc2ktf/trunk/TestingFramework $> ./run_test.sh --- Starting test for: eq2eq --- Actor: eq2eq - OK Skipping log removal --- Finished test for: eq2eq --- --- Starting test for: eq2eqmpi --- Actor: eq2eqmpi - OK Skipping log removal --- Finished test for: eq2eqmpi --- --- Starting test for: mpi --- Actor: mpi - OK Skipping log removal --- Finished test for: mpi --- --- Starting test for: nocpo --- Actor: nocpo - OK Skipping log removal --- Finished test for: nocpo --- --- Starting test for: nocpo_param --- Actor: nocpo_param - OK Skipping log removal --- Finished test for: nocpo_param --- --- Starting test for: stringinout --- Actor: stringinout - OK Skipping log removal --- Finished test for: stringinout ---
then a test report will appear inside each directory:
./eq2eqmpi/test.log ./mpi/test.log ./eq2eq/test.log ./stringinout/test.log ./nocpo/test.log ./nocpo_param/test.log
3. 3. Integrating new actor with Testing Framework
3.1. 3.1. Minimal Test Case
It is possible to integrate new actor with FC2K Testing Framework, however, it requires following a convention. Basically, the most simple test can provide only one script
run_test.sh
This test is responsible for running and reporting status of test execution.
3.2. 3.2. Set of tests provided by CPT
Default set of tests, provides testing of common cases based on FC2K and Kepler. There are few steps that each tests performs. If you want to make your own tests, I suggest to perform the same steps.
- create input data - either create input pulse file - or copy it from location - compile native code (actor) - create actor - based on your own FC2K project - when preparing FC2K project, make sure to use relative paths (e.g.: "./libactor.so") - run workflow - you need your own workflow that runs actor - if you don't want to use Kepler, you need to provide another means of running native code (e.g.: Python based wrapper) - compare results with expected ones - you have to provide means of determining whether execution was correct or not - you need to provide data set (expected) that will be compared with actual output - you should report whether test passed or failed - you should clean up after execution
3.3. 3.3. Example of Test Case - based on MPI
Let's take a look at one of Test Cases provided by FC2K Testing Framework. It consist of few files that serve given purpose.
These files are:
- FC2K projects
- Kepler workflows
- expected output
- Make file
- execution script
mpi |-- expected_output.txt - this file contains expected output of execution |-- fc2k_nocpo_mpi_batch.xml - actor that will be started on batch nodes |-- fc2k_nocpo_mpi_host.xml - actor that will be started on Gateway machine |-- fc2k_nocpo_mpi_open_batch.xml - actor that will use OpenMP in batch mode |-- fc2k_nocpo_mpi_open_host.xml - actor that will use OpenMP on Gateway |-- Makefile - provides recipe for building actor's library |-- nocpompi.f90 - source code of actor |-- run_test.sh - executes tests and reports the outcome `-- workflow_mpi_chain.xml - workflow that runs all the actors
3.4. 3.4. Developing simple execution script
Below, you can find minimal test case with all the steps required to run it
#!/bin/bash
# ../common.sh contains helper functions
source ../common.sh
# init is a wrapper for calling make (building actor library)
init_test
# we are building all the actors here
${FC2K}/bin/fc2k fc2k_nocpo_mpi_batch.xml >> test.log 2>&1
${FC2K}/bin/fc2k fc2k_nocpo_mpi_host.xml >> test.log 2>&1
${FC2K}/bin/fc2k fc2k_nocpo_mpi_open_batch.xml >> test.log 2>&1
${FC2K}/bin/fc2k fc2k_nocpo_mpi_open_host.xml >> test.log 2>&1
# start_kepler is a helper function defined in ../common.sh
# it starts workflow
start_kepler workflow_mpi_chain.xml
# this is the place where checking results occurs. Note that it is up to developer
# to define what does it mean to pass/fail the test
diff redirect/workflow_mpi_chain.Display.txt expected_output.txt >> test.log 2>&1
# status_code is a helper function that shows final result of test
# based on error code from previous call
status_code mpi false
3.5. 3.5. Common functions
FC2K Testing Framework provides set of helper functions that make it simpler to create run_test.sh script.
status_code - this function takes two arguments: name of the test and boolean argument
- name is used to show the name of the test next to execution status
- boolean flag is used to determine whether logs should be removed or not
- this function uses lats exit code ($?) to determine whether test passed or failed
init_test - this function takes no arguments. It simply runs:
make clean
make
recreates directory "redirect"
start_kepler - takes one parameter - workflow name
starts workflow