...
Note that you need to pass workflow as well. This is related to the fact that you can have multiple actors (inside workflow) that are actually pointing to the very same actor's code inside Kepler.
Replacing Batch Submission Scripts inside workfow
Whenever actor is put on canvas it takes all the parameters from the actor that is currently installed inside Kepler. This may lead to divergence between workflow and Kepler installation. Typical use-case where you may experience issues:
- create actor for Gateway
- put MPI (batch) based actor on canvas
- transfer workflow to JET
- there is an issue - Gateway's based submission system differs from JET's, so does the content of workflow (embedded batch submission script).
With new release of workflow-editor it's possible to update workflow such way it reflects the batch script from Kepler installation you want to use.
| Code Block |
|---|
> module load workflow-editor/rc
> ./bin/workflow-editor \
-updatebatch \
-actor nocpompibatch \
-workflow `pwd`/workflow.xml \
-output workflow_out.xml
11:41:16.558 [main] INFO pl.psnc.kepler.CLIParse - Parsing XML file
11:41:16.591 [main] INFO pl.psnc.kepler.CLIParse - Creating Kepler based workflow
11:41:16.943 [main] INFO pl.psnc.kepler.commands.CommandUpdateActorBatchScript - I have found batch script for actor: nocpompibatch |
there is also a possibility to pass batch script from the file. In this case, you can set the value inside workflow to whatever you like. This is useful in case you have some custom settings inside batch script.
| Code Block |
|---|
> module load workflow-editor/rc
> ./bin/workflow-editor \
-updatebatch \
-actor nocpompibatch \
-batchfile `pwd`/new_script.txt \
-workflow `pwd`/workflow.xml \
-output workflow_out.xml
11:41:16.558 [main] INFO pl.psnc.kepler.CLIParse - Parsing XML file
11:41:16.591 [main] INFO pl.psnc.kepler.CLIParse - Creating Kepler based workflow
11:41:16.943 [main] INFO pl.psnc.kepler.commands.CommandUpdateActorBatchScript - I have found batch script for actor: nocpompibatch |
Helper script for listing all "broken" actors inside workflow file
...
- diff functionality for XML files (comparison of XML inside workflow and XML inside actor)
- storing new XML inside workflow
- at the moment, XML stored inside workflow is always treated as most recent one
- in case changes inside XSD/XML, in the actor, we may end up with situation where XML stored inside workflow is incompatible with required format
- we can't easily update this information (note that there were different solutions suggested but no final decision was taken)
- it's possible to use Kepler's parsing engine to update information inside workflow "as it would be modified by Kepler
DONE: there will be new feature that will allow to "inject" new code parameters into workflow
Code Block workflow-editor -updatexml file.xml -workflow ~/workflow_ets.xml -actor name_of_the_actor
- DONE: replacing XML inside workflow with one taken from actor (similar to step above, but source of the xml file will be different)
- DONE: validation of workflow's XML against actor's XSD
- this feature will allow us to determine whether code parameters inside workflow make any sense when it comes to actor's requirements
- it will allow to verify workflow's compatibility against different releases of Kepler + actors
- error handling
- at the moment, errors are not handled in nice, user friendly way - i need to fix that
- proper handling of missing actor inside workflow
- proper handling of missing actor inside Kepler
- proper handling of actor that was not generated with new FC2K features (extracted code parameters)
- helper scripts
- DONE: listing all the actors that are "broken"
- DONE: extracting XSD/XML/Default XML from workflow in human readable format (all at once)
- adding feature that will allow to analyze Kepler installaion
- list all ITM/IMAS based actors
- validate all the actors inside Kepler
- removing XSD/Default XML from workflow
- at the moment, situation is very unclear when it comes to code parameters - we are mixing sources of information
- Code Parameters (modified by user) are taken from Workflow
- XSD is always taken from actor
- Code Parameters for new actor (dropped on canvas) are taken from actor's code
- this may lead to situation where two instances of the same actor have completely different sets of parameters
- The only reasonable thing we should keep in workflow are modified code parameters (we have now ability to validate them)
- It makes no sense to keep XSD
- It makes no sense to keep Default XML
- We can introduce XML validation into workflow execution
- Once workflow is started, each and every actor that has code parameters can validate whether parameters make sense
- In case parameters are not valid we can inform user that something fishy is going on
- we need further modifications of FC2K so we can mix up actors
- at the moment, workflow-editor is not able to determine whether actor supports new functionalities or not - it blindly assumes that actors were generated using new FC2K
- i need to extend actors such way workflow-editor is capable of checking whether actor supports CodeParameters in new fashion or not
...