Table of Contents

Introduction

DEADLINE: end of March

Objectives:

• GET: To retrieve subset of data elements
• GET_SIZE: To interrogate the size of subset of data elements,

Assumptions:

• GET_SLICE type methods are not in the scope of the task (they are alternative ways to retrieve parts of the data elements) – although may be useful to add (independently of the deliverable) if straightforward to implement.
• Implementation in Python only, no plans for other languages
• In case of nested AoS, it is not allowed to specify set of indices for AoS ancestors
• IMPLEMENTATION IN NEW LL ONLY!!!!!

...

USE CASES:

...

1. During the partial GET, only the requested node is returned (and the structure below if the node is not a leaf), not the whole structure above: get (« magnetics.flux_loop(i).flux ») à flux(t) for index (i) of its parent
2. If the user specifies a set of indices for the parent, then the GET returns a reshaped leaf with an additional dimension (corresponding to the one of its parent). Reshaping is done while keeping time as the last dimension, to keep consistent with the IMAS DD rule (even if the reshaped leaf is not anymore a legal DD entity) :
   1. get (« magnetics.flux_loop(i1:i2).flux ») à flux(i1:i2, t)
   2. get (« equilibrium.time_slice(t1:t2).profiles_1d.q ») à q(:, t1:t2)
   3. In case of nested AoS, it is not allowed to specify set of indices for AoS ancestors. Only given values of AoS ancestors indices are handled : path/to/field/with/ancestorAoS(x:y)/field/AoS(n :m) is not managed by partial GET

Type of Field	Indices	Query	Output

 

primitive/structure	NO	path/to/field/with/ancestorAoS(x)/field	primitive_type[1] structure[1]

 

primitive/structure	YES	path/to/field/with/ancestorAoS(i1:i2)/field	primitive_type[n] structure[n]	n=i2 - i1
primitive/structure	YES	path/to/field/with/ancestorAoS(x:y)/field/AoS(n :m)/field	ERROR	it is not allowed to specify set of indices for AoS ancestors
AoS	NO	path/to/field/with/ancestorAoS(x)/field/AoS(y)	AoS[1]	if AoS is not a leaf, its "sub-structure" is filled in
AoS	YES	path/to/field/with/ancestorAoS(x)/field/AoS(i1:i2)	AoS[n]	n=i2 - i1 if AoS is not a leaf, its "sub-structure" is filled in
AoS	YES	path/to/field/with/ancestorAoS(x:y)/field/AoS(n :m)	ERROR	it is not allowed to specify set of indices for AoS ancestors
AoS	YES	path/to/field/with/ancestorAoS(x:y)/field/AoS(1)	ERROR?????

Time issue in heterogeneous mode:

  Let's assume Aos is tuple {data, time}

• GET(/path/to/AoS/) - OK
• GET(/path/to/AoS/data)  
  •   Homogeneous mode: OK
  • Heterogeneous  mode:
    • If requesting for single index - OK
      
    • otherwise - ERROR
• GET(/path/to/AoS/time) 
  
  • Homogeneous mode: ids%time
  • Heterogeneous  mode: ERROR 
  
  

Example:

• get (« magnetics.flux_loop(i1:i2).flux ») returns flux(i1:i2). This is an AoS with data(t) and time(t) below, which can be of different sizes for the various indices i1:i2

• get (« magnetics.flux_loop(i1:i2).flux.data ») returns data(i1:i2,t) in homogeneous case, and an error message otherwise.

...

• ids_get(<path_including_indices>)
  input parameterspulseCtx (OR shot, run, user, tokamak, version)
• IDS name
• occurence
QUERY

• output parameters
  • array of primitive types (if query points to field of primitive type)
  • array of structures (if query points to field of AoS type or structure)
• query syntax 
  
  • separators ( / . | ) - path within tree
  • brackets ()  []   - array
  • indices (fortran style)
    • ":" all elements
    • "x:y" (from x to y)
    • ":y" from begin to y
    • "x:" from x to the last 
  • only the last bracket AoS can contain indices:
    • YES: "path/to/field/with/ancestorAoS(1)/field/AoS(n:m)"
    • NO: "path/to/field/with/ancestorAoS(x:y)/field/AoS(1)"

Requested changes:

• Low Level:
  • read_data - sanity check of type only when requested
• HLI:
  • method for creating AoS (method of classes representing AoS array) OR resize(1)
  • method to obtain AoS ctx
  • AoS classes - a special field indicating that class is AoS 

Algorithm:

Input: path (   /a/b/c(1)/d/e(m:n)/f/g  )

1) Convert path to array/list  {'a','b','c(1)','d','e(m:n)','f','g'}

2) Call traverse(ctx = p-fCtx, path, node = ids)

3) Get node name from list, look for IDS field of this name

4) Does node name contain brackets?

• NO - cut node from list, call traverse (reducedList, currentNode)
• YES - it is AoS   ,
  •  call createAoS from class representing AoS
    • call resize for a number of structures defined by indices
    • get AoS context (aos context - info about index? is ctx accessible? )
  • cut node from list
  • create AoS ctx in LL 
  • traverse (reducedList, aosNode)

OPEN POINTS:

...

path/to/field/with/ancestorAoS(1)/field/AoS(n:m) -> AoS(n:m) 

ancestorAoS(x:y)/field(:) --> field(:,x:y)

Time issues

AoS

 time independent data (no AoS3 on path)

members of timed AoS (AoS3)

•  single index from the ancestor AoS is requested, so no reshaping should be done, and no attempt to fill in time arrays should be done. 
• ERROR (because of  lack of time info available in returned data)

 time dependent vectors/arrays

Shape of returned arrays

/path/with/AoS(i1:i2)/data

lets assume:

+ i1=1; i2=3 => number of AoS to be found ("range") is 2

+ len (data) == 5,

+ data for AoS(i1) = {11, 12, 13, 14, 15}

+ data for AoS(i2) = {21, 22, 23, 24, 25}

Non timed AoS

get (« magnetics.flux_loop(i1:i2).flux ») à flux(i1:i2, t)

If AoS is NON TIMED (AoS1 or AoS2) return array has shape (range, data_len)

In NumPy notation:

[[11, 12, 13, 14, 15]

[21, 22, 23, 24, 25]]

e.g.

arr(0,1) = 12

arr(1,2) = 23

Timed AoS

get (« equilibrium.time_slice(t1:t2).profiles_1d.q ») à q(:, t1:t2)

If AoS is TIMED (AoS3) return array has shape (data_len, range)

In NumPy notation:

[ [11,21]

 [12,22]

 [13,23]

 [14,24]

 [15,25] ]

e.g.

arr(0,1) = 21

arr(2,1) = 23

OPEN POINTS:

• AoS as ancestor(s) of queried data
  • once implemented mechanism for creation of AoS could be reused to "go through" ancestors
• returned value 
  • type provided by user?
  • AoS? if not - functionality can be placed at the Low Level and available for all HLIs (however it could be a problem with returned type  anyway)
• Level of nesting
  • path/AoS(indices)/x/y/z ?

path/to/field/with/ancestorAoS(1)/field/AoS(n:m) -> AoS(n:m) 

ancestorAoS(x:y)/field(:) --> field(:,x:y)

...