Table of Contents
Introduction
DEADLINE: end of March
Objectives:
- GET: To retrieve subset of data elements
- GET_SIZE: To interrogate the size of subset of data elements,
Assumptions:
GET_SLICEtype methods are not in the scope of the task (they are alternative ways to retrieve parts of the data elements) – although may be useful to add (independently of the deliverable) if straightforward to implement.- Implementation in Python only
- Q: Any implementations in other languages in future? Should we design a common API for all languages?
Syntax:
- ids_get(<path_including_indices>)
- ids.path.to.field.get(indices) will be also investigated
query
syntax- TBD
- what users exactly need?
- we may take a look at Demux actor
Info | ||
---|---|---|
| ||
The ualdemux actor extracts single fields from a CPO given as input. Each exported field is defined by an output port added by the user, you can add as many output ports as you want. It is mandatory that the name of the port is the full path of the field to export, for example datainfo/whatref/machine is the path to machine field of a limiter CPO. The actor takes a time as input, which is propagated as output port if we manipulate a non-timed CPO, or a timed CPO in timeSlice mode. The timeSlice mode is a parameter of the actor, you can change its value by double clicking on it: if it is set at true, data are extracted for the slice corresponding at the input time, if it is set at false, data are extracted for each time slice (so a simple scalar data will become a 1D vector). If timeSlice is false, the output time becomes also a vector of times. Note that at this time, ualdemux actor allows only 1D data export in kepler. The last feature of this actor is the selection of subset to be extracted. You need to specify this selection in the name of the output port as follow: first you have to add a ; (semicolon) after the name of the field, then you have to specify the dimension in which you consider the data (1 is the first dim). After that, you can add as many ;index as you want (a semicolon followed by an index you want to extract). You can also replace the index by an expression index1:index2 (with a colon) to specify a set of indexes between both index1 and index2. For example, position/z;1;5;10;15:18 will export indexes [5,10,15,16,17,18] of the field z. |
Functionality:
- A simple structure or a leaf with no more than one array of structure ancestor,
- Returned: an array of the field i, containing the desired indices of its parent array of structure (optional argument, valid only if the requested leaf has the same size for all desired indices).
- Example : magnetics.flux_loop(indices).flux
- Q: how can we specify this with Bartek’s method ? Does a Python method accept a syntax such as magnetics.flux_loop(:).flux.get or magnetics.flux_loop.flux.get(:) ?
- An array of structure
- Returned: an array of structure containing the desired indices (optional argument, by default all indices are returned).
- Example : magnetics.flux_loop(indices) à straightforward with Bartek’s method
An array of structure with an array of structure ancestor:- Example. ic_antennas.antenna(indices1).module.get(indices2)
- It will be investigated if the method can be also easily implemented (i.e. the result is not guaranteed and the need is less obvious)
A field or a simple structure with more than one array of structure ancestor- Example: ic_antennas.antenna(indices1).module(indices2).power_launched.get
- It will be investigated if the method can be also easily implemented (i.e. the result is not guaranteed and the need is less obvious)
OPEN POINTS:
- Syntax - TBD
- Is it possible to implement ids.path.to.field.get(indices) -> arrays of structures as a part of path....
- , no plans for other languages
- In case of nested AoS, it is not allowed to specify set of indices for AoS ancestors
- IMPLEMENTATION IN NEW LL ONLY!!!!!
...
USE CASES:
- During the partial GET, only the requested node is returned (and the structure below if the node is not a leaf), not the whole structure above: get (« magnetics.flux_loop(i).flux ») à flux(t) for index (i) of its parent
- If the user specifies a set of indices for the parent, then the GET returns a reshaped leaf with an additional dimension (corresponding to the one of its parent). Reshaping is done while keeping time as the last dimension, to keep consistent with the IMAS DD rule (even if the reshaped leaf is not anymore a legal DD entity) :
- get (« magnetics.flux_loop(i1:i2).flux ») à flux(i1:i2, t)
- get (« equilibrium.time_slice(t1:t2).profiles_1d.q ») à q(:, t1:t2)
- In case of nested AoS, it is not allowed to specify set of indices for AoS ancestors. Only given values of AoS ancestors indices are handled : path/to/field/with/ancestorAoS(x:y)/field/AoS(n :m) is not managed by partial GET
Type of Field | Indices | Query | Output | |
---|---|---|---|---|
primitive/structure | NO | path/to/field/with/ancestorAoS(x)/field | primitive_type[1] structure[1] | |
primitive/structure | YES | path/to/field/with/ancestorAoS(i1:i2)/field | primitive_type[n] structure[n] | n=i2 - i1 |
primitive/structure | YES | path/to/field/with/ancestorAoS(x:y)/field/AoS(n :m)/field | ERROR | it is not allowed to specify set of indices for AoS ancestors |
AoS | NO | path/to/field/with/ancestorAoS(x)/field/AoS(y) | AoS[1] | if AoS is not a leaf, its "sub-structure" is filled in |
AoS | YES | path/to/field/with/ancestorAoS(x)/field/AoS(i1:i2) | AoS[n] | n=i2 - i1 if AoS is not a leaf, its "sub-structure" is filled in |
AoS | YES | path/to/field/with/ancestorAoS(x:y)/field/AoS(n :m) | ERROR | it is not allowed to specify set of indices for AoS ancestors |
AoS | YES | path/to/field/with/ancestorAoS(x:y)/field/AoS(1) | ERROR????? |
Time issue in heterogeneous
mode:
Let's assume Aos is tuple {data, time}
- GET(/path/to/AoS/) - OK
- GET(/path/to/AoS/data)
- Homogeneous mode: OK
- Heterogeneous mode:
- If requesting for single index - OK
- otherwise - ERROR
- If requesting for single index - OK
- GET(/path/to/AoS/time)
- Homogeneous mode: ids%time
- Heterogeneous mode: ERROR
Example:
- get (« magnetics.flux_loop(i1:i2).flux ») returns flux(i1:i2). This is an AoS with data(t) and time(t) below, which can be of different sizes for the various indices i1:i2
- get (« magnetics.flux_loop(i1:i2).flux.data ») returns data(i1:i2,t) in homogeneous case, and an error message otherwise.
...
- ids_get(<path_including_indices>)
- output parameters
- array of primitive types (if query points to field of primitive type)
- array of structures (if query points to field of AoS type or structure)
- output parameters
query
syntax- separators ( / ) - path within tree
- brackets () - array
- indices (fortran style)
- ":" all elements
- "x:y" (from x to y)
- ":y" from begin to y
- "x:" from x to the last
- only the last AoS can contain indices:
- YES: "path/to/field/with/ancestorAoS(1)/field/AoS(n:m)"
- NO: "path/to/field/with/ancestorAoS(x:y)/field/AoS(1)"
path/to/field/with/ancestorAoS(1)/field/AoS(n:m) -> AoS(n:m)
ancestorAoS(x:y)/field(:) --> field(:,x:y)
Time issues
AoS
time independent data (no AoS3 on path)
members of timed AoS (AoS3)
- single index from the ancestor AoS is requested, so no reshaping should be done, and no attempt to fill in time arrays should be done.
- ERROR (because of lack of time info available in returned data)
time dependent vectors/arrays
Shape of returned arrays
/path/with/AoS(i1:i2)/data
lets assume:
+ i1=1; i2=3 => number of AoS to be found ("range") is 2
+ len (data) == 5,
+ data for AoS(i1) = {11, 12, 13, 14, 15}
+ data for AoS(i2) = {21, 22, 23, 24, 25}
Non timed AoS
get (« magnetics.flux_loop(i1:i2).flux ») à flux(i1:i2, t)
If AoS is NON TIMED (AoS1 or AoS2) return array has shape (range, data_len)
In NumPy notation:
[[11, 12, 13, 14, 15]
[21, 22, 23, 24, 25]]
e.g.
arr(0,1) = 12
arr(1,2) = 23
Timed AoS
get (« equilibrium.time_slice(t1:t2).profiles_1d.q ») à q(:, t1:t2)
If AoS is TIMED (AoS3) return array has shape (data_len, range)
In NumPy notation:
[ [11,21]
[12,22]
[13,23]
[14,24]
[15,25] ]
e.g.
arr(0,1) = 21
arr(2,1) = 23
OPEN POINTS:
- AoS as ancestor(s) of queried data
- once implemented mechanism for creation of AoS could be reused to "go through" ancestors
- returned value
- type provided by user?
- AoS? if not - functionality can be placed at the Low Level and available for all HLIs (however it could be a problem with returned type anyway)
- Level of nesting
- path/AoS(indices)/x/y/z ?
...