Versions Compared

Old Version 16

changes.mady.by.user Bartosz Palak

Saved on

compared with

New Version 17

changes.mady.by.user Bartosz Palak

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
This page describes the important API differences of the IMAS Access Layer version 4 (AL4), especially for users which have already experience with previous version 3 (AL3). 
This page is intended as a quick HOWTO, for more details on the AL4 usage please refer to the "AL user guide" (todo: link the user guide here when available from imas.iter.org).

 

Info
titleData Dictionary dependency

IMAS modules and versions are composed of both a Data Dictionary version (DD) and the Access Layer version (AL). Access Layer 4.0.0 was released the 29th of March 2019 for the 3.21.1 DD version. While in principle older DD version can be supported, for practical reasons no older DD are planned to be released with AL4. DD 3.21.0 is the last version supported by AL3, all more recent versions can only be released with AL4.

 

Below you will a section on main difference in high-level interface API. Then users API for each high-level language will be compared side by side comparison for AL3 and AL4. todo: add section on backends

 

Main discrepancies

Info

Names of functions given hereafter might differ slightly in the language specific high-level interfaces, please refer to language specific sections below or to "Access Layer user's guide" for exact names and signatures, as well as for differences which are specific to a given language interface.

  • Reference to a data-entry now requires always to specify user, machine and version, as there is no proper default mechanism. This means that open(),create() functions are now deprecated, and only open_env() and create_env() remain in AL4.
  • Side effect of the change above, calling imasdb is not mandatory anymore (it was with AL3 for using either open(),create() or Kepler), but the script is still present in order to help users to create the local database layout (one time call per machine/version).
  • put_non_timed() function is now deprecated, as this feature was not used when not followed by at least one put_slice(). AL3 chain of put_non_timed()+put_slice() should be translated into a single call to put() where IDS contains only one slice of timed data. Note that this combined approach is valid in both AL3 and AL4, and that put_slice() has a well defined behavior only when the passed IDS contains one single slice of timed data.
  • The caching mechanism, as well as the flush(),discard() operations are deprecated. A new memory backend is introduced in AL4 in order to reduce IO costs of disk accesses when not necessary.
  • New API to select any available backend will be added soon, in the meantime it is possible to use the lowlevel API (if accessible from targeted language interface) to perform this selection (currently available backends are MDSplus, Memory and UDA). 

 

 

Tip
titleMDSplus pulsefiles conversion

The MDSplus models (structure of the pulsefiles) have changed a lot between AL3 and AL4. Reading with AL4 a pulsefile written with AL3 or the other way around is not possible. Nevertheless a conversion script is provided with any AL4 release, so users can attempt to convert an AL3 generated pulse into a AL4 compatible one. This script is developed purely in MDSplus and will work only for pulsefiles with IDS in homogeneous_time and which are well formed. Usage, after having loaded one AL4 IMAS module (or imasenv <= 3.21.1) is as the following:

Code Block
ualport.py <shot> <run> <oldDir> <newDir>
	<shot>   : shot number, or 'all' for converting all shots presents in <oldDir>
	<run>    : run number, or 'all' for converting all runs presents in <oldDir>
	<oldDir> : directory containing the source (AL3) pulsefile
	<newDir> : directory where will be stored the destination (converted to AL4) pulsefile

For sake of clarity, it is strongly advised that you use two separate directories (machine names for instance) to differenciate AL3 and AL4 pulses. For instance you can rename your test tokamak (or any other machine name) to test-AL3 and recreate an empty test where AL4 generated or converted pulsefiles will be stored:

Code Block
mv ~/public/imasdb/test ~/public/imasdb/test-AL3
imasdb test
ualport.py 123 1 ~/public/imasdb/test-AL3/3/0 ~/public/imasdb/test/3/0

n.b. "3" in the oldDir and newDir path of the example above are not referring to AL3! This stands for the major version of the DD (which is identical at the moment).

 

 

 

Fortran

  • bindings to lowlevel C/C++ is done explicitly through iso_c_bindings, without reallocating arrays when possible (for all non-character based quantities)

    Warning
    Because of arrays being allocated in C when IDS is obtained from AL (get/get_slice), deallocation / allocation should be done with extra care especially for routines using IDS as inout arguments! A new substructure level deallocation routine was introduced (see table below) but it needs additional logical input specifying if arrays of the substructure were allocated in C (AL) or Fortran (user's code). Two options:
    a) the IDS is deallocate within the user's code scope, then a mix of C/F allocation need to be deallocated accordingly
    b) the IDS is potentially used/extended/deallocated by another code, or is coming from different contexts (AL or another Fortran code), then the safer approach is to work on a copy of the IDS (potentially only for inout IDS which are coming from AL, which can be checked using the new is_c_data routine presented in table below)
  • wrappers around lowlevel API are provided, allowing for instance to select a different backend

 

OperationAL3AL4Remarks

creating a data-entry

(MDSplus backend)

call imas_create(name,shot,run,shotref,runref,idx)
call imas_create_env(name,shot,run,shotref,runref,& 
                     idx,user,machine,version)

 

 

call imas_create_env(name,shot,run,shotref,runref,&
                     idx,user,machine,version)

Concept of default machine is deprecated

Note

Arguments name,refshot,refrun can have any (or empty) value, and are kept only for compatibility. A new unique data-entry selection routine will be added to the API, making this routine deprecated at term.

opening a data-entry

(MDSplus backend)

call imas_open(name,shot,run,idx)
call imas_open_env(name,shot,run,idx,&
                   user,machine,version)
 
call imas_open_env(name,shot,run,idx,&
                   user,machine,version)

Concept of default machine is deprecated

Note

Argument name can have any (or empty) value, and is kept only for compatibility. A new unique data-entry selection routine will be added to the API, making this routine deprecated at term.

writing the complete IDS
call ids_put(idx,idsname,ids)
call ids_put(idx,idsname,ids)
 
writing a single slice of an IDS
call ids_put_slice(idx,idsname,ids)
call ids_put_slice(idx,idsname,ids)
 
writing only non-timed data of an IDS
call ids_put_non_timed(idx,idsname,ids)
 Deprecated, replaced by ids_put() on IDS containing only first slice data
reading the complete IDS
call ids_get(idx,idsname,ids)
call ids_get(idx,idsname,ids)
 
reading a single slice of an IDS
call ids_get_slice(idx,idsname,ids,time,interpol)
call ids_get_slice(idx,idsname,ids,time,interpol)
Paremeters are defined in ual_defs module for the different interpolation modes (to be used for interpol argument): CLOSEST_INTERP,PREVIOUS_INTERP,LINEAR_INTERP
copying an IDS or one of its substructure
call ids_copy(src,dest)
call ids_copy(src,dest)

 

 

 

deallocating an IDS or one if its substructure
call ids_deallocate(ids)
call ids_deallocate(ids)
call ids_deallocate_struct(struct,c_data)
When deallocating a substructure of an IDS, one need to pass a logical argument indicating whether this structure has been obtained allocated in C (obtained from AL get(),get_slice()) or not. This check is performed automatically while deallocating an entire IDS
checking if arrays of an IDS were allocated in C (by AL) 
call is_c_data(ids,c_data)
Returns the c_data logical which is .true. if the IDS was obtained through the Access Layer, .false. otherwise.
deleting an IDS from a data-entry
call ids_delete(idx,idsname,ids)
call ids_delete(idx,idsname,ids)
 

closing a data-entry

call imas_close(idx)
call imas_close(idx)
 

 

 

 

C++

 

OperationAL3AL4Remarks

creating a data-entry

(MDSplus backend)

Create IDS object IdsNs::IDS ids(pulse,run,0,0);

then create() on the newly created object

e.g.

IdsNs::IDS ids(12,1,0,0);

ids.create()

 

Create IDS object IdsNs::IDS ids(pulse,run,0,0); then

createEnv(UserName, DatabaseName, IMASVersion); on

newly created object

e.g.

IdsNs::IDS ids(12,1,0,0);

ids.createEnv(‘usr’, ‘test’, ‘3’);

Create IDS object IdsNs::IDS ids(pulse,run,0,0); then createEnv(UserName, DatabaseName, IMASVersion); on newly created object

e.g.

IdsNs::IDS ids(12,1,0,0);

ids.createEnv(‘usr’, ‘test’, ‘3’);

 

 

opening a data-entry

(MDSplus backend)

Create IMAS entry object IdsNs::IDS

imas_entry(pulse,run,0,0); then open() on the

newly created object

e.g.

IdsNs::IDS imas_entry(12,1,0,0);

imas_entry.open();

 

Create IDS object IdsNs::IDS

imas_entry(pulse,run,0,0); then

openEnv(UserName, DatabaseName,

IMASMajorVersion) on the newly created object

e.g.

IdsNs::IDS imas_entry(12,1,0,0);

imas_entry.openEnv(‘usr’, ‘test’, ‘3’);

 

Create IDS object IdsNs::IDS imas_entry(pulse,run,0,0); then openEnv(UserName, DatabaseName, IMASMajorVersion) on the newly created object

e.g.

IdsNs::IDS imas_entry(12,1,0,0);

imas_entry.openEnv(‘usr’, ‘test’, ‘3’);

 

writing the complete IDS
 

Method of the ids object: def put(self, occurrence=0)

Example:

IdsNs::IDS imas_entry(12,1,0,0);

imas_entry._pf_active.ids_properties.comment_of

= "Test data";

imas_entry._pf_active.put(); for occurrence 0

imas_entry._pf_active.put(n); for occurrence n

Method of the ids object: def put(self, occurrence=0)

Example:

IdsNs::IDS imas_entry(12,1,0,0);

imas_entry._pf_active.ids_properties.comment_of = "Test data";

imas_entry._pf_active.put(); for occurrence 0

imas_entry._pf_active.put(n); for occurrence n
 
writing a single slice of an IDS
 

Method of the ids object: def putSlice(self, occurrence=0)

Example:

IdsNs::IDS imas_entry(12,1,0,0);

imas_entry._actuator.putSlice();for occurrence 0

imas_entry._actuator.putSlice(n); for occurrence n

 
writing only non-timed data of an IDS
 
Deprecated 
reading the complete IDS
 

Method of the ids def get(self, occurrence=0)

Example:

IdsNs::IDS ids(12,1,0,0);

ids.open(); //Open the database

ids._pf_active.get(); for occurrence 0

ids._pf_active.get(n); for occurrence n

Method of the ids def get(self, occurrence=0)

Example:

IdsNs::IDS ids(12,1,0,0);

ids.open(); //Open the database

ids._pf_active.get(); for occurrence 0

ids._pf_active.get(n); for occurrence n

 
reading a single slice of an IDS
 

Method of the ids def getSlice(self, occurrence=0,

time_requested, interpolation_method)

Example:

IdsNs::IDS ids(12,1,0,0);

ids._pf_active.getSlice(time_requested,

interpolation_method); for occurrence 0

ids._pf_active.getSlice(n, time_requested,

interpolation_method); for occurrence n

 

Method of the ids def getSlice(self, occurrence=0, time_requested, interpolation_method)

Example:

IdsNs::IDS ids(12,1,0,0);

ids._pf_active.getSlice(time_requested, interpolation_method); for occurrence 0

ids._pf_active.getSlice(n, time_requested, interpolation_method); for occurrence n
CLOSEST_SAMPLE PREVIOUS_SAMPLE INTERPOLATION
copying an IDS or one of its substructure
 
 
 
flush an IDS from the cache to the disk    
flushing all IDSes from the cache to the disk   
deallocating an IDS or one if its substructure
 
  
checking if arrays of an IDS were allocated in C (by AL) 
 
 
deleting an IDS from a data-entry
 

Method remove of the ids

e.g.

ids._pf_active.remove()

 

Method remove of the ids

e.g.

ids._pf_active.remove()
 

closing a data-entry

 

Method close() of the imas_entry object.

e.g.

IdsNs::IDS imas_entry(12,1,0,0);

imas_entry.close();

def close(self)

e.g.

imas_entry.close()

 

Method close() of the imas_entry object.

e.g.

IdsNs::IDS imas_entry(12,1,0,0);

imas_entry.close();
 

 

 

 

Python

 

OperationAL3AL4Remarks

creating a data-entry

(MDSplus backend)

A sequence of calls imas.ids(pulse, run) and

create() on newly created object

e.g.

import imas

ids_object = imas.ids(12,1)

ids_object.create()

sequence of calls imas.ids(pulse, run) and

e.g.

import imas

ids_object = imas.ids(12,1)

ids_object.create_env(‘usr’, ‘test’, ‘3’)

sequence of calls imas.ids(pulse, run) and

e.g.

import imas 

ids_object = imas.ids(12,1)

ids_object.create_env(‘usr’, ‘test’, ‘3’) idx = imas_create_env( treename, pulse, run, 0, 0, UserName, DatabaseName,IMASMajorVersion )

 

 

opening a data-entry

(MDSplus backend)

A sequence of calls imas.ids(pulse, run) and open()

on newly created object

e.g.

import imas

imas_entry = imas.ids(12,1,0,0)

imas_entry.open()

sequence of calls imas.ids(pulse, run) and

open_env(UserName, DatabaseName,

IMASMajorVersion) on newly created object

e.g.

import imas

imas_entry = imas.ids(12,1)

 

A sequence of calls imas.ids(pulse, run) and open_env(UserName, DatabaseName, IMASMajorVersion) on newly created object

e.g.

import imas

imas_entry = imas.ids(12,1)

imas_entry.open_env(‘usr’, ‘test’, ‘3’)

 

writing the complete IDS
 

Method of the imas_entry object: def put(self,

occurrence=0)

e.g.

imas_entry.pf_active.put() for occurrence 0, or

imas_entry.pf_active.put(occurrence) otherwise

Method of the imas_entry object: def put(self, occurrence=0)

e.g.

imas_entry.pf_active.put() for occurrence 0, or imas_entry.pf_active.put(occurrence) otherwise
 
writing a single slice of an IDS
 
 

Method of the ids: def putSlice(self, occurrence=0):

e.g.

imas_entry.pf_active.putSlice() for occurrence 0, or imas_entry.pf_active.putSlice(occurrence) otherwise
 
writing only non-timed data of an IDS
 

Method of the ids: def putNonTimed(self,

occurrence=0):

e.g.

imas_entry.pf_active.putNonTimed() for

occurrence 0, or

imas_entry.pf_active.putNonTimed(occurrence)

otherwise

Deprecated 
reading the complete IDS
 

def get(self, occurrence=0):

e.g.

imas_entry.pf_active.get() for occurrence 0, or

imas_entry.pf_active.get(occurrence) otherwise

 

def get(self, occurrence=0):

e.g.

imas_entry.pf_active.get() for occurrence 0, or imas_entry.pf_active.get(occurrence) otherwise
 
reading a single slice of an IDS
 

Method of the ids: def getSlice(self,

time_requested, interpolation_method,

occurrence=0):

e.g.

imas_entry.pf_active.getSlice(0.1, 2) for

occurrence 0, or

imas_entry.pf_active.getSlice(0.1, 2,

occurrence) otherwise

 

Method of the ids:  def getSlice(self, time_requested, interpolation_method, occurrence=0):

e.g.

imas_entry.pf_active.getSlice(0.1, 2) for occurrence 0, or imas_entry.pf_active.getSlice(0.1, 2, occurrence) otherwise

imasdef.py:

UNDEFINED_INTERP

CLOSEST_ INTERP

PREVIOUS_INTERP  

LINEAR_INTERP  
copying an IDS or one of its substructure
 
 
  
flush an IDS from the cache to the disk    
flushing all IDSes from the cache to the disk   
deallocating an IDS or one if its substructure
 N/A
N/A 
deleting an IDS from a data-entry
 

Method of the ids: def deleteData(self,

occurrence=0):

e.g.

imas_entry.pf_active.deleteData() for

occurrence 0, or

imas_entry.pf_active.deletaData(occurrence)

otherwise

 

Method of the ids: def deleteData(self, occurrence=0):

e.g.

imas_entry.pf_active.deleteData() for occurrence 0, or imas_entry.pf_active.deletaData(occurrence) otherwise
 

closing a data-entry

 
 

def close(self)

e.g.

imas_entry.close()
 

 

 

Matlab

 

OperationAL3AL4Remarks

creating a data-entry

(MDSplus backend)

idx = imas_create(treename, pulse, run)

idx = imas_create_env( treename, pulse, run, 0, 0,

UserName, DatabaseName,IMASMajorVersion )

idx = imas_create_env( treename, pulse, run, 0, 0, UserName, DatabaseName,IMASMajorVersion )

 

 

opening a data-entry

(MDSplus backend)

idx = imas_create_env( treename, pulse, run, 0, 0,

idx = imas_create(treename, pulse, run)UserName, DatabaseName,IMASMajorVersion )

 

  idx = imas_open_env( treename, pulse, run, UserName, DatabaseName, IMASMajorversion )

 

writing the complete IDS
 ids_put(idx, IDSOccurrence, IDSVariable)

ids_put(idx, IDSOccurrence, IDSVariable)

 
writing a single slice of an IDS
 

ids_put_slice(idx, IDOccurrence,

IDSVariable)

  ids_put_slice(idx, IDOccurrence, IDSVariable)
 
writing only non-timed data of an IDS

ids_put_non_timed(idx, IDSOccurence,

IDSVariable)

Deprecated 
reading the complete IDS
 

IDSVariable = ids_get(expIdx,

IDSOccurrence)

  IDSVariable = ids_get(expIdx, IDSOccurrence)
 
reading a single slice of an IDS
 

IDSVariable = ids_get_slice(idx,

IDSOccurrence, time_requested,

interpolation_method)

  IDSVariable = ids_get_slice(idx, IDSOccurrence, time_requested, interpolation_method)
 
copying an IDS or one of its substructure
 
 
  
flush an IDS from the cache to the disk    
flushing all IDSes from the cache to the disk   
deallocating an IDS or one if its substructure
 N/A
N/A 
deleting an IDS from a data-entry
 Not implemented yet
  Not implemented yet
 

closing a data-entry

 [ cr ] = imas_close(idx)
  [ cr ] = imas_close(idx)
 

 

 

Java

 

OperationAL3AL4Remarks

creating a data-entry

(MDSplus backend)

imas.create(treename, pulse, run, 0, 0), for

example :

idx = imas.create(“ids” , 12, 1, 0, 0) ;

imas.createEnv (treename, pulse, run, 0, 0,

UserName, DatabaseName, IMASMajorVersion), for example :

idx = imas.createEnv(“ids”, 12, 1, 0, 0, “user”,

“test”, “3”) ;

imas.createEnv (treename, pulse, run, 0, 0, UserName, DatabaseName, IMASMajorVersion), for example :

idx = imas.createEnv(“ids”, 12, 1, 0, 0, “user”, “test”,  “3”) ;

 

 

opening a data-entry

(MDSplus backend)

Create IMAS entry object IdsNs::IDS

imas_entry(pulse,run,0,0); then open() on the

newly created object

e.g.

IdsNs::IDS imas_entry(12,1,0,0);

imas_entry.open();

Create IDS object IdsNs::IDS

imas_entry(pulse,run,0,0); then

openEnv(UserName, DatabaseName,

IMASMajorVersion) on the newly created object

e.g.

IdsNs::IDS imas_entry(12,1,0,0);

imas_entry.openEnv(‘usr’, ‘test’, ‘3’);

imas.openEnv(treename, pulse, run, UserName, DatabaseName, IMASVersion), for example :

idx = imas.openEnv(“ ids”, 12, 1, “user”, “test”,  “3”) ;

 

writing the complete IDS
 

Method of the ids: put (idx, IDSOccurrence,

IDSVariable), for example :

imas.equilibrium equilibrium = new

imas.equilibrium();

equilibrium.put(idx, “equilibrium”,

equilibrium) ;

Method of the ids: put (idx, IDSOccurrence, IDSVariable), for example :

imas.equilibrium equilibrium = new imas.equilibrium();

equilibrium.put(idx, “equilibrium”,  equilibrium) ;
 
writing a single slice of an IDS
 

Method of the ids: putSlice (idx, IDSOccurrence,

IDSVariable), for example :

imas.equilibrium equilibrium = new

imas.equilibrium();

equilibrium. putSlice(idx, “equilibrium”,

equilibrium) ;

 

Method of the ids: putSlice (idx, IDSOccurrence, IDSVariable), for example :

imas.equilibrium equilibrium = new imas.equilibrium();

equilibrium. putSlice(idx, “equilibrium”, equilibrium) ;
 
writing only non-timed data of an IDS
 

Method of the ids: putNonTimed (idx,

IDSOccurrence, IDSVariable), for example :

imas.equilibrium equilibrium = new

imas.equilibrium();

equilibrium.putNonTimed(idx, “equilibrium”,

equilibrium) ;

Deprecated 
reading the complete IDS
 

Method of the ids: get (idx, IDSOccurrence,

IDSVariable), for example :

imas.equilibrium equilibrium = new

imas.equilibrium();

equilibrium. get(idx, “equilibrium”,

equilibrium) ;

 

Method of the ids: get (idx, IDSOccurrence, IDSVariable), for example :

imas.equilibrium equilibrium = new imas.equilibrium();

equilibrium. get(idx, “equilibrium”,  equilibrium) ;
 
reading a single slice of an IDS
 
 

Method of the ids: getSlice (idx, IDSOccurrence, IDSVariable, double time_requested, int interpolation_method), for example :

imas.equilibrium equilibrium = new imas.equilibrium();

equilibrium.getSlice(idx, “equilibrium”,  equilibrium, 0.1, 2) ;
 
copying an IDS or one of its substructure
 
 
  
flushing all IDSes from the cache to the disk    
flushing all IDSes from the cache to the disk   
deallocating an IDS or one if its substructure
 N/A
N/A 
deleting an IDS from a data-entry
 

Method of the ids: delete (idx, IDSOccurrence,

IDSVariable), for example :

imas.equilibrium equilibrium = new

imas.equilibrium();

equilibrium.delete(idx, “equilibrium”,

equilibrium) ;

 

Method of the ids: delete (idx, IDSOccurrence, IDSVariable), for example :

imas.equilibrium equilibrium = new imas.equilibrium();

equilibrium.delete(idx, “equilibrium”,  equilibrium) ;
 

closing a data-entry

 
  imas.close(idx) ;
 

 

 

Lowlevel (C)