1. Use cases:

1.1. Wrapper modes:

interactive - user code is run directly from wrapper
batch - user executable is sent to a queue

1.2. Execution modes

User code can be run in mode:

Sequential
Parallel:
- MPI
  - MPICH
  - OpenMPI
- and/or OpenMP

1.3. Debugging modes:

standalone - debugger launches executable containing user code in a separate process
attach - debugger attaches to running process executing user code

1.4. Sandbox

"Sandbox" - a directory, in which actor will be run. Before execution of user codes wrapped by FC2K generated actor, directory will be changed to "sandbox", and after actor finishes, current directory will be switched back to previous value. The name (path) of "sandbox" directory will be created automatically or specified by user in actor configuration dialog.

Actor will use existing directory or will create it, if directory not exists.

Typical usage: checkpointing, caching intermediate results, usage of additional information (input, config files) not provided in workflow.

2. Wrapper API - current status

def actor_name(integer0, integer1, double0, double1, core_profiles0, core_profiles1, equilibrium0, exec_type='mpi_local', mpi_process = 4, strip_output = True):

2.1. Input arguments required by user C++/F method

Number and an order of arguments must be exactly the same as in user method
Types of arguments
- IDS
- Primitive type
- Array of primitive types
- String

2.2. Execution type

Named argument (exec_type='').
Current values of exec_type :
- ctypes - interactive execution (yes - this name is REALLY CONFUSING)
- mpi_local - run as MPI job
- dbg - to keep compatibility with obsolete 'debug' argument of wrapper

2.3. Additional arguments

Auxiliary keyword arguments:

mpi_processes - integer - number of MPI processes
lib_opt - boolean - determines of alternative user library should be used, instead of the default one
get_output_idss - boolean - determines, what actually will be returned if returned argument is IDS: an IDS object, or metadata describing it
strip_output - boolean - it determines (if returned argument is a string) it it should be stripped or not
dbgmode - boolean - unknown purpose

2.4. Returned values

Output values:

None - if nothing is returned from called F/C++ method
A single object - if called method returns only one value
A tuple of objects - if called methods returns more than one value. It contains also diagnostic_info if being used.

3. Weaknesses of current solution

very long list of arguments - it is easy to make a mistake:
- providing incorrect number of arguments
- providing incorrect order of arguments
lack of important information, so many features are not handled:
- no debug mode
- MPI parameters - only number of nodes is specified
- no batch mode

4. Wrapper API - new design

def actor_name(code_args = None, xml_parameters = None, job_settings = None):
	return None.  

# no returned values!

4.1. Assumptions:

General:

Python version 3.6+
Programming conventions (PEP 8?)
- https://www.python.org/dev/peps/pep-0008/
NO BACKWARD COMPATIBILITY - Incompatibility with existing Python actors!
All input arguments (user method arg and info determining how to launch job ) 'structured' within classes
Classes will be defined for:
- user code arguments - autogenerated
- XML input parameters
- job settings
The order of arguments will be arbitrary - implemented as arguments list (*args) or keyword arguments list (**kwargs)
All arguments will be optional. If object of given class will be not provided - default values will be used.

4.2. Benefits of proposed solution

Just a few arguments passed to wrapper (and not several)
Information passed to wrapper is well structured (classes keeping all arguments and settings)
Proposed solution prevents from errors resulting from incorrect order/type/number of arguments
List of arguments can be easily extended - no API changes required

4.3. Arguments required by user C++/F method

ToolName

All code samples below use <ToolName> as a prefix of classes. Once we decide on the name, all the classes will contain it as a part of the name in each class that is not actor dependent.

All arguments 'packed' within a class <ActorName>Arguments
Class will be automatically generated, and copied to a wrapper package
Order of attributes corresponds to order of user function arguments
Every attribute is a class <ToolName>Argument keeping not only value but also metadata describing argument:
- Name
- Type of value
- Value
- IN or OUT

# AUTO GENERATED !
class <ActorName>Arguments : 
  def __init__(self):
    self.arg1 = <ToolName>Argument( 'InVar', int, None, IN)
    self.arg2 = <ToolName>Argument( 'OutVar', double, None, OUT)
    self.diagnostic_info = <ToolName>DiagnosticInfo()

# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
class <ToolName>Argument :
  def __init__(self, name, type, sub_type, intent):
    self.name =  name #'argument name' READ ONLY
	self.type = type # (INT, DOUBLE, STRING, COMPLEX, IDS) READ ONLY
	self.sub_type = sub_type  #'equilibrium'  for IDSes only READ ONLY
	self.intent = intent # (IN/OUT) READ ONLY
	self.value  = 7. # to be set by user
    
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
class <ToolName>DiagnosticInfo :
  def __init__(self):
    self.status  
	self.message

4.4. XML parameters of user C++/F method

Not passed as wrapper argument
Defined (path to XML file) at the time of actor creation by <ToolName>

class <ToolName>XMLParamaters :  
  def __init__(self):
    self.parameters = # file name or string or... (?)
    self.default_parameters = # file name or string or ... (?)
	self.schema = # file name or string or ... (?)

4.5. Job settings

class <ToolName>JobSettings : 
  def __init__(self):
    self.batch_job = <ToolName>BatchJob()
    self.debug = <ToolName>Debug()
    self.mpi = <ToolName>MPI()
	self.open_mp = <ToolName>OpenMP()
	self.sandbox = <ToolName>Sandbox()
	self.?????  #any other info needed?

# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
class <ToolName>BatchJob :
  def __init__(self):
	self.queue = 
	self.TBD

# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
class <ToolName>Debug :
  def __init__(self):
	self.debugger #TotalView/gdb
	self.mode     #attach/standalone
	self.TBD

# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
class <ToolName>MPI :
  def __init__(self):
	self.TBD
	self.debug_switch

# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
class <ToolName>OpenMP :
  def __init__(self):
	self.TBD

# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
class <ToolName>Sandbox :
  def __init__(self):
	self.path
	self.lifetime
	self.TBD

4.6. Wrapper outcome

OUT arguments - wrapper will update fields of <ActorName>Arguments class

5. Example

from my_test.wrapper import my_test_actor

# # # # # JOB SETTINGS # # # # # # # 
job_settings = <ToolName>JobSettings()
 
#batch job
job_settings.batch_job.queue = 'gw_default"
#debugging
# no debugging for batch jobs
#mpi
job_settings.mpi.nodes = 5
#OpenMP
# it is not OpenMP job
# Sandbox
job_settings.sandbox.lifetime = LIFTIME_WORKFLOW
job_settings.sandbox.clean_up = True

# # # # # CODE ARGUMENTS # # # # # # # 
arguments = <ActorName>Arguments()  

self.my_in_arg_01.value = 5
self.my_in_arg_02.value = 0.123
self.my_in_core_profiles_01.value = in_cp_obj
self.my_in_core_profiles_02.value = in_cp_metadata #object keeping shot/run/user

# # # # # RUNNING ACTOR # # # # # # # 
my_test(code_args = arguments, job_settings = job_settings)

# # # # # RETURN # # # # # # # 
ret_eq = arguments.my_out_equilibrium_02.value

6. Open points

Only IN and OUT arguments (no INOUT arguments)
Arrays as an inout of user method- only "dynamic" - i.e. of variable size
Wrapper results:
1. OUT arguments - wrapper will update fields of <ActorName>Arguments class
2. INCOMPATIBILITY
Diagnostic info
1. Info returned from user method
  1. status flag
  2. user defined message
2. 'Q: can it be mandatory in user sbrt?
Sandbox:
1. Do we need this feature?
Alternative library:
1. Do we need this feature?

Page tree

FC2K Python wrapper redesign