1. Use cases:
1.1. Wrapper modes:
- interactive - user code is run directly from wrapper
- batch - user executable is sent to a queue
1.2. Execution modes
User code can be:
- Sequential
- Parallel:
- MPI
- MPICH
- OpenMPI
- and/or OpenMP
- MPI
1.3. Debugging modes:
- standalone - debugger launches executable containing user code in a separate process
- attach - debugger attaches to running process executing user code
1.4. Sandbox
"Sandbox" - a directory, in which actor will be run. Before execution of user codes wrapped by FC2K generated actor, directory will be changed to "sandbox", and after actor finishes, current directory will be switched back to previous value. The name (path) of "sandbox" directory will be created automatically or specified by user in actor configuration dialog.
Actor will use existing directory or will create it, if directory not exists.
Typical usage: checkpointing, caching intermediate results, usage of additional information (input, config files) not provided in workflow.
2. Wrapper API - current status
def actor_name(integer0, integer1, core_profiles0, exec_type='ctypes', **kwargs):
2.1. Input arguments required by user C++/F method
- Number and an order of arguments must be exactly the same as in user method
- Types of arguments
- IDS
- Primitive type
- Array of primitive types
- String
2.2. Execution type
- Named argument (
exec_type='')
. - Current values of
exec_type
:ctypes
- interactive execution (yes - this name is REALLY CONFUSING)- mpi_local - run as MPI job
dbg
- to keep compatibility with obsolete 'debug' argument of wrapper
2.3. Additional arguments
Auxiliary keyword arguments:
mpi_processes
- integer - number of MPI processeslib_opt
- boolean - determines of alternative user library should be used, instead of the default oneget_output_idss
- boolean - determines, what actually will be returned if returned argument is IDS: an IDS object, or metadata describing itstrip_output
- boolean - it determines (if returned argument is a string) it it should be stripped or notdbgmode
- boolean - unknown purpose
2.4. Returned values
Output values:
None
- if nothing is returned from called F/C++ method- A single object - if called method returns only one value
- A tuple of objects - if called methods returns more than one value. It contains also
diagnostic_info
if being used.
3. Weaknesses of current solution
- very long list of arguments
- it is easy to make a mistake providing incorrect number of arguments
- it is easy to make a mistake providing incorrect order of arguments
- lack of important information, so many features are not handled:
- no debug mode
- MPI parameters - only number of nodes is specified
- no batch mode
4. Wrapper API - new design
4.1. Assumptions:
- All input arguments (user method arg and info determining how to launch job ) 'structured' within classes
- Classes will be defined for:
- user code arguments - autogenerated
- XML input parameters
- diagnostic info
- execution modes
- return values - autogenerated
- The order of arguments (classes) will be arbitrary - implemented as arguments list (*args) or keyword arguments list (**kwargs)
- All arguments (classes) will be optional. If object of given class will be not provided - default values will be used.
4.2. Benefits of proposed solution
- Just a few arguments passed to wrapper (and not several)
- Information passed to wrapper is well structured (classes keeping all arguments and settings)
- Proposed solution prevents from errors resulting from incorrect order/type/number of arguments
4.3. Arguments required by user C++/F method
- All arguments 'packed' within a class <ActorName>Arguments
- Class will be automatically generated within a wrapper package
- Order of attributes corresponds to order of user function arguments
- Every attribute is a class FC2PyArgument keeping not only value but also metadata describing argument:
- Name
- Type of value
- Value
- IN or OUT
- Attributes' setters/getters will be overridden - user will set only value of argument
- To reduce complexity visible by user
- To check if arg type is correct
4.4. XML parameters of user C++/F method
- Not passed as wrapper argument
- Defined (path to XML file) at the time of actor creation by FC2Py
4.5. Job settings
- A class passed as wrapper arguments
- Tree (of classes) describing job settings
- FC2PyJobSettings class:
- batch_job - class FC2PyBatchJob:
- queue
- ???
- debug - class FC2PyDebug
- debugger - TotalView/gdb
- mode - attach/standalone
- mpi - class FC2PyMPI
- mpi parameters - TBD
- ???
- open_mp - class FC2PyOpenMP
- openMP parameters - TBD
- ???
- sandbox
- path to sandbox
- batch_job - class FC2PyBatchJob:
4.6. Wrapper outcome
- To be discussed -
- PREFERRED: OUT arguments - wrapper will update fields of <ActorName>Arguments class
- Returned values - packed within a class:
- autogenerated
- a part of wrapper package
5. Open points
- Programming conventions (PEP 8?)
- Incompatibility with existing Python actors!
- Only IN and OUT arguments (no INOUT arguments)
- Arrays as an inout of user method- only "dynamic" - i.e. of variable size
- Wrapper results:
- OUT arguments - wrapper will update fields of <ActorName>Arguments class
- Returned values - packed within a class
- Diagnostic info
- Info returned from user method
- status flag
- user defined message
- 'Q: can it be mandatory in user sbrt?
- Info returned from user method
- Sandbox:
- Do we need this feature?
- Alternative library:
- Do we need this feature?