Workflow Editor

Workflow Editor allows to analyse the workflow's (e.g. ETS) Code Parameters.

...

Code Block
> module load workflow-editor

Getting help

To get basic overview of available options call workflow-editor with help argument

Code Block

> workflow-editor --help
usage: workflow-editor
   -help                  print this message
   -actors                list all actors with Code Parameters
   -version               print the version information and exit
   -workflow <file>       analyze this workflow file
   -output <file>         name of the output workflow
   -actor <actorname>     name of the actor we will retrieve
   -xml                   list Code Parameters for actor
   -dxml                  list default Code Parameters for actor
   -xsd                   list XSD definition of Code Parameters for actor
   -validate <dest>       validate with [workflow|kepler]
   -actorxml              show XML Code Parameters inside actor
   -actorxsd              show XSD Code Parameters inside actor
   -actordxml             show Default XML Code Parameters inside actor

Listing actors inside workflow

You can get the list of all actors that have Code Parameters by calling workflow-editor with "-actors" argument (remember to pass workflow as well)


   -updatexml             update XML inside workflow using actor's version
   -updatexsd             update XSD inside workflow using actor's version
   -updatedxml            update default XML inside workflow using actor's version

Listing actors inside workflow

You can get the list of all actors that have Code Parameters by calling workflow-editor with "-actors" argument (remember to pass workflow as well)

Code Block

> workflow-editor -actors -workflow workflow_ets.xml

Code Block

> workflow-editor -actors -workflow workflow_ets.xml
12:29:25.267 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - WEILAND - eu.itm.TCIWEILAND.TCIWEILAND
12:29:25.269 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - emeq_inside_simple_init - eu.itm.emeq.emeq
12:29:25.269 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - addICant - eu.itm.addICant.addICant
12:29:25.269 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - spider_inside_consistent_init - eu.itm.spider.spider
12:29:25.270 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - TGLF - eu.itm.TCITGLF.TCITGLF
12:29:25.270267 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - cheaseWEILAND - eu.itm.cheaseTCIWEILAND.cheaseTCIWEILAND
12:29:25.270269 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - TORAYFOMemeq_inside_simple_init - eu.itm.TORAYFOMemeq.TORAYFOMemeq
12:29:25.270269 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - synchrotron_cytranaddICant - eu.itm.synchrotron_cytran.synchrotron_cytran.addICant.addICant
12:29:25.270269 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - GLF23spider_inside_consistent_init - eu.itm.TCIGLF23spider.TCIGLF23spider
12:29:25.270 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - QLKTGLF - eu.itm.TCIQLKTCITGLF.TCIQLKTCITGLF
12:29:25.270 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - icdeptchease - eu.itm.icdeptchease.icdeptchease
12:29:25.270 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - EDWMTORAYFOM - eu.itm.TCIEDWMTORAYFOM.TCIEDWMTORAYFOM
12:29:25.270 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - afsisynchrotron_cytran - eu.itm.afsi.afsisynchrotron_cytran.synchrotron_cytran
12:29:25.270 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - ntmetsGLF23 - eu.itm.ntmetsTCIGLF23.ntmetsTCIGLF23
12:29:25.270 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - SOURCE_COMBINERQLK - eu.itm.sourcecombinerTCIQLK.sourcecombinerTCIQLK
12:29:25.270 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - impuritymanipulatoricdept - eu.itm.impuritymanipulatoricdept.impuritymanipulatoricdept
12:29:25.270 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - ascot4parallelEDWM - eu.itm.ascot4parallelTCIEDWM.ascot4parallelTCIEDWM
12:29:25.270 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - emeq_inside_consistent_initafsi - eu.itm.emeqafsi.emeqafsi
12:29:25.270 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - NCLASSntmets - eu.itm.TCINCLASSZntmets.TCINCLASSZntmets
12:29:25.270 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - toricSOURCE_COMBINER - eu.itm.toricsourcecombiner.toricsourcecombiner
12:29:25.271270 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - EDWMZimpuritymanipulator - eu.itm.TCIEDWMZimpuritymanipulator.TCIEDWMZimpuritymanipulator
12:29:25.271270 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - nemoascot4parallel - eu.itm.nemoascot4parallel.nemoascot4parallel
12:29:25.271270 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - MMMemeq_inside_consistent_init - eu.itm.TCIMMMemeq.TCIMMMemeq
12:29:25.271270 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - lionNCLASS - eu.itm.LIONsliceTCINCLASSZ.LIONsliceTCINCLASSZ
12:29:25.271270 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - pellettriggertoric - eu.itm.pellettriggertoric.pellettriggertoric
12:29:25.271 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - cyrano-in-loopEDWMZ - eu.itm.CyranoTCIEDWMZ.CyranoTCIEDWMZ
12:29:25.271 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - spotnemo - eu.itm.spotnemo.spotnemo
12:29:25.271 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - teffcvgMMM - eu.itm.teffcvgTCIMMM.teffcvgTCIMMM
12:29:25.271 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - deltacombinerlion - eu.itm.deltacombinerLIONslice.deltacombinerLIONslice
12:29:25.271 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - StixReDistpellettrigger - eu.itm.StixReDistpellettrigger.StixReDistpellettrigger
12:29:25.271 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - ascot4serialcyrano-in-loop - eu.itm.ascot4serialCyrano.ascot4serialCyrano
12:29:25.271 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - nuclearsimspot - eu.itm.nuclearsimspot.nuclearsimspot
12:29:25.271 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - sawcrashteffcvg - eu.itm.sawcrashteffcvg.sawcrashteffcvg
12:29:25.271 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - spider_inside_simple_initdeltacombiner - eu.itm.spiderdeltacombiner.spiderdeltacombiner
12:29:25.271 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - IMP4impStixReDist - eu.itm.IMP4impStixReDist.IMP4impStixReDist
12:29:25.271 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - nbisimascot4serial - eu.itm.nbisimascot4serial.nbisimascot4serial
12:29:25.271 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - hcd2coresourcenuclearsim - eu.itm.hcd2coresourcenuclearsim.hcd2coresourcenuclearsim
12:29:25.271 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - graysawcrash - eu.itm.graysawcrash.graysawcrash
12:29:25.272271 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - RELAXspider_inside_simple_init - eu.itm.RELAXspider.RELAXspider
12:29:25.272271 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - gaussiansourcesIMP4imp - eu.itm.gausiansourcesIMP4imp.gausiansourcesIMP4imp
12:29:25.272271 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - nbisim2nbisim - eu.itm.nbisim2nbisim.nbisim2nbisim
12:29:25.272271 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - RITMhcd2coresource - eu.itm.TCIRITMhcd2coresource.TCIRITMhcd2coresource
12:29:25.272271 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - addECantgray - eu.itm.addECantgray.addECantgray
12:29:25.272 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - checktearingRELAX - eu.itm.checktearingRELAX.checktearingRELAX
12:29:25.272 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - hcd2corefastgaussiansources - eu.itm.hcd2corefastgausiansources.hcd2corefastgausiansources
12:29:25.272 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - pelletactornbisim2 - eu.itm.pelletactornbisim2.pelletactornbisim2
12:29:25.272 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - sawupdateRITM - eu.itm.sawupdateTCIRITM.sawupdateTCIRITM
12:29:25.272 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - cyranoaddECant - eu.itm.CyranoaddECant.CyranoaddECant
12:29:25.272 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - ntmDeffTLetschecktearing - eu.itm.ntmDeffTLetschecktearing.ntmDeffTLetschecktearing
12:29:25.272 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - cecehcd2corefast - eu.itm.cecehcd2corefast.cecehcd2corefast
12:29:25.272 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - sawcritpelletactor - eu.itm.sawcritpelletactor.sawcritpelletactor
12:29:25.272 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - chease_inside_consistent_initsawupdate - eu.itm.cheasesawupdate.cheasesawupdate
12:29:25.272 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - nbifillercyrano - eu.itm.nbifillerCyrano.nbifillerCyrano
12:29:25.272 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - NEOntmDeffTLets - eu.itm.TCINEOntmDeffTLets.TCINEOntmDeffTLets
12:29:25.272 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - riskcece - eu.itm.riskcece.riskcece
12:29:25.273272 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - emeqsawcrit - eu.itm.emeqsawcrit.emeqsawcrit
12:29:25.273272 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - travischease_inside_consistent_init - eu.itm.travischease.travischease
12:29:25.273272 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - spidernbifiller - eu.itm.spidernbifiller.spidernbifiller
12:29:25.273272 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - chease_inside_simple_initNEO - eu.itm.cheaseTCINEO.cheaseTCINEO
12:29:25.273272 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - BgB_JETTO_Particlesrisk - eu.itm.BgB_JETTO_Particles.BgB_JETTO_Particlesrisk.risk
12:29:25.273 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - emeq lion-in-loop- eu.itm.emeq.emeq
12:29:25.273 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - travis - eu.itm.LIONslice.LIONslice

Retrieving XML/Default XML/XSD for actor

Once you have the name of the actor, you can list it's Code Parameters

Code Block

# User's Code Parameters
> workflow-editor -xml -actor risk -workflow workflow_ets.xml
<?xml version="1.0" encoding="UTF-8"?>
<?jaxfront version=2.66;time=2014-10-28 17:14:54.425?>
<parameters xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
       <n_out_profiles>20</n_out_profiles>
       <dx_resolution>0.02</dx_resolution>
       <fac_implicit>0.7</fac_implicit>
       <debug_mode>0</debug_mode>
</parameters>

travis.travis
12:29:25.273 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - spider - eu.itm.spider.spider
12:29:25.273 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - chease_inside_simple_init - eu.itm.chease.chease
12:29:25.273 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - BgB_JETTO_Particles - eu.itm.BgB_JETTO_Particles.BgB_JETTO_Particles
12:29:25.273 [main] INFO  pl.psnc.kepler.commands.CommandGetActors - lion-in-loop - eu.itm.LIONslice.LIONslice

Retrieving XML/Default XML/XSD for actor

Once you have the name of the actor, you can list it's Code Parameters

Code Block

# User's Code Parameters
> workflow-editor -xml -actor risk

Code Block

# Default Code Parameters
> workflow-editor -dxml -actor risk -workflow workflow_ets.xml
<?xml version="1.0" encoding="UTF-8"?>
<parameters>
    <!-- Resolution of the output 1D-profiles -->
<?jaxfront version=2.66;time=2014-10-28 17:14:54.425?>
<parameters xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
       <n_out_profiles> 30 <profiles>20</n_out_profiles>
    <!-- Normalized velocity resolution (v/vth) -->
    <dx_resolution> 0.01 <resolution>0.02</dx_resolution>
      <!-- Implicit/explicit contribution for integration scheme (Crank-Nicholson or so) -->
 <fac_implicit>0.7</fac_implicit>
       <fac_implicit> 0.7 </fac_implicit><debug_mode>0</debug_mode>
</parameters>

Code Block

# XSD based definition ofDefault Code Parameters
> workflow-editor -xsddxml -actor risk -workflow workflow_ets.xml
<?xml version="1.0" encoding="UTF-8"?>
 <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
    elementFormDefault="qualified">
   <xs:annotation><parameters>
    <!-- Resolution of the output 1D-profiles -->
    <n_out_profiles> 30 </n_out_profiles>
     <xs:documentation>Code parameters for Risk (Fokker-Planck code)</xs:documentation>
   </xs:annotation>
   <xs:simpleType name="integer_minInclusive_2"><!-- Normalized velocity resolution (v/vth) -->
    <dx_resolution> 0.01 </dx_resolution>
     <xs:restriction base="xs:integer">
       <xs:minInclusive value="2"/<!-- Implicit/explicit contribution for integration scheme (Crank-Nicholson or so) -->
    <fac_implicit> 0.7 </fac_implicit>
</parameters>

Code Block

# XSD based definition of Code Parameters
> workflow-editor -xsd -actor risk -workflow workflow_ets.xml
<?xml version="1.0" encoding="UTF-8"?>
 <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
    elementFormDefault="qualified">
   <xs:annotation>
     <xs:documentation>Code parameters for Risk (Fokker-Planck code)</xs:documentation>xs:restriction>
   </xs:simpleType>
   <xs:simpleType name="float_minInclusive_dot0001">
     <xs:restriction base="xs:float">
       <xs:minInclusive value="0.0001"/>
     </xs:restriction>
   </xs:simpleType>annotation>
   <xs:simpleType name="floatinteger_minInclusive_dot12">
     <xs:restriction base="xs:floatinteger">
       <xs:minInclusive value="0.12"/>
     </xs:restriction>
   </xs:simpleType>
   <!-- document element -->
<xs:simpleType name="float_minInclusive_dot0001">
     <xs:elementrestriction namebase="parametersxs:float">
       <xs:complexType>minInclusive value="0.0001"/>
       <xs:all></xs:restriction>
      </xs:simpleType>
   <xs:elementsimpleType refname="nfloat_outminInclusive_profiles" minOccurs="0"/dot1">
         <xs:elementrestriction refbase="dx_resolution" minOccurs="0"/xs:float">
         <xs:element ref="fac_implicit" minOccursminInclusive value="0.1"/>
       </xs:all>restriction>
     </xs:complexType>simpleType>
   </xs:element>
   <!-- Resolution of the output 1D profiles document element -->
   <xs:element name="n_out_profiles" type="integer_minInclusive_2"parameters">
      <xs:annotation>complexType>
          <xs:documentation>Resolution of the output 1D profiles</xs:documentation>
all>
         </xs:annotation>
   </xs:element><xs:element ref="n_out_profiles" minOccurs="0"/>
   <!-- Normalized velocity resolution (v/vth) -->
   <xs:element nameref="dx_resolution" typeminOccurs="float_minInclusive_dot00010"/>
      <xs:annotation>
   <xs:element ref="fac_implicit" minOccurs="0"/>
     <xs:documentation>Normalized velocity resolution (v/vth)</xs:documentation>all>
      </xs:annotation>complexType>
   </xs:element>
   <!-- Implicit/explicitResolution contributionof forthe integrationoutput scheme (Crank-Nicholson or so)1D profiles -->
   <xs:element name="facn_out_implicitprofiles" type="floatinteger_minInclusive_dot12">
      <xs:annotation>
          <xs:documentation>Implicit/explicitdocumentation>Resolution contributionof forthe integrationoutput scheme (Crank-Nicholson or so)<1D profiles</xs:documentation>
      </xs:annotation>
   </xs:element>
   </xs:schema>

You can combine command (e.g. get everything at once)

Code Block
workflow-editor -xsd -xml -dxml -actor risk -workflow workflow_ets.xml

Getting Code Parameters from actor (installed inside Kepler)

It may happen that Code Parameters inside actor's code (inside Kepler) differ from data stored inside workflow. This can happen whenever you try to open workflow with more recent version of Kepler.

You can retrieve information about Code Parameters inside actor by calling workflow-editor with -actorxml/-actordxml/-actorxsd.

Info
Note that getting information from actors requires recompilation of the actor. Getting information regarding Code Parameters is a new feature in FC2K and will not work for old actors (e.g. imported into Kepler using import_actor).

Code Block

> workflow-editor -actorxsd -actor nocpoparam -workflow workflow.xml
<?xml version="1.0" encoding="utf-8"?>
<!-- RFOF code parameters XML schema -->
<xs:schema
  xmlns:xs="http://www.w3.org/2001/XMLSchema"
  elementFormDefault="qualified">
   <xs:simpleType name="FloatList">
      <xs:list itemType="xs:float"/>
   </xs:simpleType>
   <xs:simpleType name="IntegerList">
      <xs:list itemType="xs:integer"/>!-- Normalized velocity resolution (v/vth) -->
   <xs:element name="dx_resolution" type="float_minInclusive_dot0001">
      <xs:annotation>
          <xs:documentation>Normalized velocity resolution (v/vth)</xs:documentation>
      </xs:annotation>
   </xs:element>
   <!-- Implicit/explicit contribution for integration scheme (Crank-Nicholson or so) -->
   <xs:element name="fac_implicit" type="float_minInclusive_dot1">
      <xs:annotation>
          <xs:documentation>Implicit/explicit contribution for integration scheme (Crank-Nicholson or so)</xs:documentation>
      </xs:annotation>
   </xs:simpleType>element>
   <xs:simpleType name="BooleanList">
      <xs:list itemType="xs:boolean"/>
   </xs:simpleType>

  <xs:element name="parameters">
    <xs:complexType>
      <xs:all>
	<xs:element ref="r"/>
	<xs:element ref="i"/>
	<xs:element ref="boolean"/>
	<xs:element ref="realVec"/>
	<xs:element ref="intVec"/>
	<xs:element ref="booleanVec"/>
      </xs:all>
    </xs:complexType>
  </xs:element>
  <xs:element name="r" type="xs:float">
    <xs:annotation>
      <xs:documentation>A real value</xs:documentation>
    </xs:annotation>
  </xs:element>
  <xs:element name="i" type="xs:integer">
    <xs:annotation>
      <xs:documentation>An integer number</xs:documentation>
    </xs:annotation>
  </xs:element>
  <xs:element name="boolean" type="xs:boolean">
    <xs:annotation>
      <xs:documentation>An integer number</xs:documentation>
    </xs:annotation>
  </xs:element>
  <xs:element name="realVec" type="FloatList">
    <xs:annotation>
      <xs:documentation>A real valued vector</xs:documentation>
    </xs:annotation>
  </xs:element>
  <xs:element name="intVec" type="IntList">
    <xs:annotation>
      <xs:documentation>An integer valued vector</xs:documentation>
    </xs:annotation>
  </xs:element>
  <xs:element name="booleanVec" type="BooleanList">
    <xs:annotation>
      <xs:documentation>An boolean valued vector</xs:documentation>
    </xs:annotation>
  </xs:element>

</xs:schema>

Note that you need to pass workflow as well. This is related to the fact that you can have multiple actors (inside workflow) that are actually pointing to the very same actor's code inside Kepler.

Helper script for listing all "broken" actors inside workflow file

It may happen that workflow contains inconsistent Code Parameters for an actor. For example, XML doesn't conform to XSD stored inside workflow. You can make bulk check of workflow by creating wrapper script.

Code Block

language	bash
title	make sure to save this script in $HOME/bulk_check.sh

#!/bin/bash

while read p; do
   echo "----- Validating actor: ${p} -----"
   ./bin/workflow-editor -validate -workflow $1 -actor ${p}
   echo "----------------------"
done <$2

To make it work, you will also need list of all actors that belong to workflow and have Code Parameters.

Code Block

# I assume that you have file "workflow_ets.xml" inside your $HOME
 
> workflow-editor -actors -workflow ~/workflow_ets.xml | cut -f2 -d "-" > ~/actors_to_check
> chmod +x $HOME/bulk_check.sh
> $HOME/bulk_check.sh $HOME/workflow_ets.xml $HOME/actors_to_check

After running this script you will get the list of all actors that have some sort of issues related to XSD vs. XML inside Code Parameters.

Further development

There are few more features that have to be developed to make the tool more useful

</xs:schema>

You can combine command (e.g. get everything at once)

Code Block
workflow-editor -xsd -xml -dxml -actor risk -workflow workflow_ets.xml

Getting Code Parameters from actor (installed inside Kepler)

It may happen that Code Parameters inside actor's code (inside Kepler) differ from data stored inside workflow. This can happen whenever you try to open workflow with more recent version of Kepler.

You can retrieve information about Code Parameters inside actor by calling workflow-editor with -actorxml/-actordxml/-actorxsd.

Info
Note that getting information from actors requires recompilation of the actor. Getting information regarding Code Parameters is a new feature in FC2K and will not work for old actors (e.g. imported into Kepler using import_actor).

Code Block

> workflow-editor -actorxsd -actor nocpoparam -workflow workflow.xml
<?xml version="1.0" encoding="utf-8"?>
<!-- RFOF code parameters XML schema -->
<xs:schema
  xmlns:xs="http://www.w3.org/2001/XMLSchema"
  elementFormDefault="qualified">
   <xs:simpleType name="FloatList">
      <xs:list itemType="xs:float"/>
   </xs:simpleType>
   <xs:simpleType name="IntegerList">
      <xs:list itemType="xs:integer"/>
   </xs:simpleType>
   <xs:simpleType name="BooleanList">
      <xs:list itemType="xs:boolean"/>
   </xs:simpleType>

  <xs:element name="parameters">
    <xs:complexType>
      <xs:all>
	<xs:element ref="r"/>
	<xs:element ref="i"/>
	<xs:element ref="boolean"/>
	<xs:element ref="realVec"/>
	<xs:element ref="intVec"/>
	<xs:element ref="booleanVec"/>
      </xs:all>
    </xs:complexType>
  </xs:element>
  <xs:element name="r" type="xs:float">
    <xs:annotation>
      <xs:documentation>A real value</xs:documentation>
    </xs:annotation>
  </xs:element>
  <xs:element name="i" type="xs:integer">
    <xs:annotation>
      <xs:documentation>An integer number</xs:documentation>
    </xs:annotation>
  </xs:element>
  <xs:element name="boolean" type="xs:boolean">
    <xs:annotation>
      <xs:documentation>An integer number</xs:documentation>
    </xs:annotation>
  </xs:element>
  <xs:element name="realVec" type="FloatList">
    <xs:annotation>
      <xs:documentation>A real valued vector</xs:documentation>
    </xs:annotation>
  </xs:element>
  <xs:element name="intVec" type="IntList">
    <xs:annotation>
      <xs:documentation>An integer valued vector</xs:documentation>
    </xs:annotation>
  </xs:element>
  <xs:element name="booleanVec" type="BooleanList">
    <xs:annotation>
      <xs:documentation>An boolean valued vector</xs:documentation>
    </xs:annotation>
  </xs:element>

</xs:schema>

Note that you need to pass workflow as well. This is related to the fact that you can have multiple actors (inside workflow) that are actually pointing to the very same actor's code inside Kepler.

Replacing Batch Submission Scripts inside workfow

Whenever actor is put on canvas it takes all the parameters from the actor that is currently installed inside Kepler. This may lead to divergence between workflow and Kepler installation. Typical use-case where you may experience issues:

create actor for Gateway
put MPI (batch) based actor on canvas
transfer workflow to JET
there is an issue - Gateway's based submission system differs from JET's, so does the content of workflow (embedded batch submission script).

With new release of workflow-editor it's possible to update workflow such way it reflects the batch script from Kepler installation you want to use.

Code Block

> module load workflow-editor/rc
> ./bin/workflow-editor \
-updatebatch \
-actor nocpompibatch \
-workflow `pwd`/workflow.xml \
-output workflow_out.xml
11:41:16.558 [main] INFO  pl.psnc.kepler.CLIParse - Parsing XML file
11:41:16.591 [main] INFO  pl.psnc.kepler.CLIParse - Creating Kepler based workflow
11:41:16.943 [main] INFO  pl.psnc.kepler.commands.CommandUpdateActorBatchScript - I have found batch script for actor: nocpompibatch

there is also a possibility to pass batch script from the file. In this case, you can set the value inside workflow to whatever you like. This is useful in case you have some custom settings inside batch script.

Code Block

> module load workflow-editor/rc
> ./bin/workflow-editor \
-updatebatch \
-actor nocpompibatch \
-batchfile `pwd`/new_script.txt \
-workflow `pwd`/workflow.xml \
-output workflow_out.xml
11:41:16.558 [main] INFO  pl.psnc.kepler.CLIParse - Parsing XML file
11:41:16.591 [main] INFO  pl.psnc.kepler.CLIParse - Creating Kepler based workflow
11:41:16.943 [main] INFO  pl.psnc.kepler.commands.CommandUpdateActorBatchScript - I have found batch script for actor: nocpompibatch

Helper script for listing all "broken" actors inside workflow file

It may happen that workflow contains inconsistent Code Parameters for an actor. For example, XML doesn't conform to XSD stored inside workflow. You can make bulk check of workflow by creating wrapper script.

Code Block

language	bash
title	make sure to save this script in $HOME/bulk_check.sh

#!/bin/bash

while read p; do
   echo "----- Validating actor: ${p} -----"
   ./bin/workflow-editor -validate -workflow $1 -actor ${p}
   echo "----------------------"
done <$2

To make it work, you will also need list of all actors that belong to workflow and have Code Parameters.

Code Block

# I assume that you have file "workflow_ets.xml" inside your $HOME
 
> workflow-editor -actors -workflow ~/workflow_ets.xml | cut -f2 -d "-" > ~/actors_to_check
> chmod +x $HOME/bulk_check.sh
> $HOME/bulk_check.sh $HOME/workflow_ets.xml $HOME/actors_to_check

After running this script you will get the list of all actors that have some sort of issues related to XSD vs. XML inside Code Parameters.

Further development

There are few more features that have to be developed to make the tool more useful

diff functionality for XML files (comparison of XML inside workflow and XML inside actor)
storing new XML inside workflow
- at the moment, XML stored inside workflow is always treated as most recent one
- in case changes inside XSD/XML, in the actor, we may end up with situation where XML stored inside workflow is incompatible with required format
- we can't easily update this information (note that there were different solutions suggested but no final decision was taken)
- it's possible to use Kepler's parsing engine to update information inside workflow "as it would be modified by Kepler
- DONE: there will be new feature that will allow to "inject" new code parameters into workflow
  Code Block
  workflow-editor -updatexml file.xml -workflow ~/workflow_ets.xml -actor name_of_the_actor
  Image Added
DONE: replacing XML inside workflow with one taken from actor (similar to step above, but source of the xml file will be different)
DONE: validation of workflow's XML against actor's XSD
- this feature will allow us to determine whether code parameters inside workflow make any sense when it comes to actor's requirements
- it will allow to verify workflow's compatibility against different releases of Kepler + actors
error handling
- at the moment, errors are not handled in nice, user friendly way - i need to fix that
- proper handling of missing actor inside workflow
- proper handling of missing actor inside Kepler
- proper handling of actor that was not generated with new FC2K features (extracted code parameters)
helper scripts
- DONE: listing all the actors that are "broken"
- DONE: extracting XSD/XML/Default XML from workflow in human readable format (all at once)
adding feature that will allow to analyze Kepler installaion
- list all ITM/IMAS based actors
- validate all the actors inside Kepler
removing XSD/Default XML from workflow
- at the moment, situation is very unclear when it comes to code parameters - we are mixing sources of information
- Code Parameters (modified by user) are taken from Workflow
- XSD is always taken from actor
- Code Parameters for new actor (dropped on canvas) are taken from actor's code
  - this may lead to situation where two instances of the same actor have completely different sets of parameters
- The only reasonable thing we should keep in workflow are modified code parameters (we have now ability to validate them)
- It makes no sense to keep XSD
- It makes no sense to keep Default XML
- We can introduce XML validation into workflow execution
  - Once workflow is started, each and every actor that has code parameters can validate whether parameters make sense
  - In case parameters are not valid we can inform user that something fishy is going on
we need further modifications of FC2K so we can mix up actors
- at the moment, workflow-editor is not able to determine whether actor supports new functionalities or not - it blindly assumes that actors were generated using new FC2K
- i need to extend actors such way workflow-editor is capable of checking whether actor supports CodeParameters in new fashion or not
diff functionality for XML files (comparison of XML inside workflow and XML inside actor)
storing new XML inside workflow
- at the moment, XML stored inside workflow is always treated as most recent one
- in case changes inside XSD/XML, in the actor, we may end up with situation where XML stored inside workflow is incompatible with required format
- we can't easily update this information (note that there were different solutions suggested but no final decision was taken)
- it's possible to use Kepler's parsing engine to update information inside workflow "as it would be modified by Kepler
- there will be new feature that will allow to "inject" new code parameters into workflow
  Code Blockworkflow-editor -updatexml file.xml -workflow ~/workflow_ets.xml -actor name_of_the_actor

Page tree

Versions Compared

Old Version 9

New Version Current

Key

Table of Contents

Workflow Editor

Getting help

Listing actors inside workflow

Listing actors inside workflow

Retrieving XML/Default XML/XSD for actor

Retrieving XML/Default XML/XSD for actor

Getting Code Parameters from actor (installed inside Kepler)

Helper script for listing all "broken" actors inside workflow file

Further development

Getting Code Parameters from actor (installed inside Kepler)

Replacing Batch Submission Scripts inside workfow

Helper script for listing all "broken" actors inside workflow file

Further development

Page tree

Page History

Versions Compared

Old Version 9

New Version Current

Key

Table of Contents

Workflow Editor

Getting help

Listing actors inside workflow

Listing actors inside workflow

Retrieving XML/Default XML/XSD for actor

Retrieving XML/Default XML/XSD for actor

Getting Code Parameters from actor (installed inside Kepler)

Helper script for listing all "broken" actors inside workflow file

Further development

Getting Code Parameters from actor (installed inside Kepler)

Replacing Batch Submission Scripts inside workfow

Helper script for listing all "broken" actors inside workflow file

Further development