This page describes the important API differences of the IMAS Access Layer version 4 (AL4), especially for users which have already experience with previous version 3 (AL3).
This page is intended as a quick HOWTO, for more details on the AL4 usage please refer to the "AL user guide" (todo: link the user guide here when available from imas.iter.org).
Data Dictionary dependency
IMAS modules and versions are composed of both a Data Dictionary version (DD) and the Access Layer version (AL). Access Layer 4.0.0 was released the 29th of March 2019 for the 3.21.1 DD version. While in principle older DD version can be supported, for practical reasons no older DD are planned to be released with AL4. DD 3.21.0 is the last version supported by AL3, all more recent versions can only be released with AL4.
Below you will a section on main difference in high-level interface API. Then users API for each high-level language will be compared side by side comparison for AL3 and AL4. todo: add section on backends
1.1. Main discrepancies
Names of functions given hereafter might differ slightly in the language specific high-level interfaces, please refer to language specific sections below or to "Access Layer user's guide" for exact names and signatures, as well as for differences which are specific to a given language interface.
- Reference to a data-entry now requires always to specify user, machine and version, as there is no proper default mechanism. This means that
open(),create()
functions are now deprecated, and onlyopen_env()
andcreate_env()
remain in AL4. - Side effect of the change above, calling
imasdb
is not mandatory anymore (it was with AL3 for using eitheropen(),create()
or Kepler), but the script is still present in order to help users to create the local database layout (one time call per machine/version). put_non_timed()
function is now deprecated, as this feature was not used when not followed by at least oneput_slice()
. AL3 chain ofput_non_timed()+put_slice()
should be translated into a single call toput()
where IDS contains only one slice of timed data. Note that this combined approach is valid in both AL3 and AL4, and thatput_slice()
has a well defined behavior only when the passed IDS contains one single slice of timed data.- The caching mechanism, as well as the
flush(),discard()
operations are deprecated. A new memory backend is introduced in AL4 in order to reduce IO costs of disk accesses when not necessary. - New API to select any available backend will be added soon, in the meantime it is possible to use the lowlevel API (if accessible from targeted language interface) to perform this selection (currently available backends are MDSplus, Memory and UDA).
MDSplus pulsefiles conversion
The MDSplus models (structure of the pulsefiles) have changed a lot between AL3 and AL4. Reading with AL4 a pulsefile written with AL3 or the other way around is not possible. Nevertheless a conversion script is provided with any AL4 release, so users can attempt to convert an AL3 generated pulse into a AL4 compatible one. This script is developed purely in MDSplus and will work only for pulsefiles with IDS in homogeneous_time and which are well formed. Usage, after having loaded one AL4 IMAS module (or imasenv >= 3.21.1) is as the following:
ualport.py <shot> <run> <oldDir> <newDir> <shot> : shot number, or 'all' for converting all shots presents in <oldDir> <run> : run number, or 'all' for converting all runs presents in <oldDir> <oldDir> : directory containing the source (AL3) pulsefile <newDir> : directory where will be stored the destination (converted to AL4) pulsefile
For sake of clarity, it is strongly advised that you use two separate directories (machine names for instance) to differenciate AL3 and AL4 pulses. For instance you can rename your test tokamak (or any other machine name) to test-AL3 and recreate an empty test where AL4 generated or converted pulsefiles will be stored:
mv ~/public/imasdb/test ~/public/imasdb/test-AL3 imasdb test ualport.py 123 1 ~/public/imasdb/test-AL3/3/0 ~/public/imasdb/test/3/0
n.b. "3" in the oldDir and newDir path of the example above are not referring to AL3! This stands for the major version of the DD (which is identical at the moment).
1.2. Fortran
bindings to lowlevel C/C++ is done explicitly through iso_c_bindings, without reallocating arrays when possible (for all non-character based quantities)
Because of arrays being allocated in C when IDS is obtained from AL (get/get_slice
), deallocation / allocation should be done with extra care especially for routines using IDS asinout
arguments! A new substructure level deallocation routine was introduced (see table below) but it needs additional logical input specifying if arrays of the substructure were allocated in C (AL) or Fortran (user's code). Two options:
a) the IDS is deallocate within the user's code scope, then a mix of C/F allocation need to be deallocated accordingly
b) the IDS is potentially used/extended/deallocated by another code, or is coming from different contexts (AL or another Fortran code), then the safer approach is to work on a copy of the IDS (potentially only forinout
IDS which are coming from AL, which can be checked using the new is_c_data routine presented in table below)- wrappers around lowlevel API are provided, allowing for instance to select a different backend
Operation | AL3 | AL4 | Remarks |
---|---|---|---|
creating a data-entry (MDSplus backend) | call imas_create(name,shot,run,shotref,runref,idx) call imas_create_env(name,shot,run,shotref,runref,&
|
call imas_create_env(name,shot,run,shotref,runref,& | Concept of default machine is deprecated Arguments |
opening a data-entry (MDSplus backend) | call imas_open(name,shot,run,idx) call imas_open_env(name,shot,run,idx,& | call imas_open_env(name,shot,run,idx,& | Concept of default machine is deprecated Argument |
writing the complete IDS | call ids_put(idx,idsname,ids) |
| |
writing a single slice of an IDS | call ids_put_slice(idx,idsname,ids) | call ids_put_slice(idx,idsname,ids) | |
writing only non-timed data of an IDS | call ids_put_non_timed(idx,idsname,ids) | Deprecated, replaced by ids_put() on IDS containing only first slice data | |
reading the complete IDS | call ids_get(idx,idsname,ids) | call ids_get(idx,idsname,ids) | |
reading a single slice of an IDS | call ids_get_slice(idx,idsname,ids,time,interpol) | call ids_get_slice(idx,idsname,ids,time,interpol) | Paremeters are defined in ual_defs module for the different interpolation modes (to be used for interpol argument): CLOSEST_INTERP,PREVIOUS_INTERP,LINEAR_INTERP |
copying an IDS or one of its substructure | call ids_copy(src,dest) | call ids_copy(src,dest) |
|
deallocating an IDS or one if its substructure | call ids_deallocate(ids) | call ids_deallocate(ids) call ids_deallocate_struct(struct,c_data) | When deallocating a substructure of an IDS, one need to pass a logical argument indicating whether this structure has been obtained allocated in C (obtained from AL get(),get_slice() ) or not. This check is performed automatically while deallocating an entire IDS |
checking if arrays of an IDS were allocated in C (by AL) | call is_c_data(ids,c_data) | Returns the c_data logical which is .true. if the IDS was obtained through the Access Layer, .false. otherwise. | |
deleting an IDS from a data-entry | call ids_delete(idx,idsname,ids) | call ids_delete(idx,idsname,ids) | |
closing a data-entry | call imas_close(idx) | call imas_close(idx) |
1.3. C++
Operation | AL3 | AL4 | Remarks |
---|---|---|---|
creating a data-entry (MDSplus backend) | Create IDS object IdsNs::IDS ids(pulse,run,0,0); then create() on the newly created object e.g. IdsNs::IDS ids(12,1,0,0); ids.create()
Create IDS object IdsNs::IDS ids(pulse,run,0,0); then createEnv(UserName, DatabaseName, IMASVersion); on newly created object e.g. IdsNs::IDS ids(12,1,0,0); ids.createEnv(‘usr’, ‘test’, ‘3’); | Create IDS object IdsNs::IDS ids(pulse,run,0,0); then createEnv(UserName, DatabaseName, IMASVersion); on newly created object e.g. IdsNs::IDS ids(12,1,0,0); ids.createEnv(‘usr’, ‘test’, ‘3’); |
|
opening a data-entry (MDSplus backend) | Create IMAS entry object IdsNs::IDS imas_entry(pulse,run,0,0); then open() on the newly created object e.g. IdsNs::IDS imas_entry(12,1,0,0); imas_entry.open();
Create IDS object IdsNs::IDS imas_entry(pulse,run,0,0); then openEnv(UserName, DatabaseName, IMASMajorVersion) on the newly created object e.g. IdsNs::IDS imas_entry(12,1,0,0); imas_entry.openEnv(‘usr’, ‘test’, ‘3’); | Create IDS object IdsNs::IDS imas_entry(pulse,run,0,0); then openEnv(UserName, DatabaseName, IMASMajorVersion) on the newly created object e.g. IdsNs::IDS imas_entry(12,1,0,0); imas_entry.openEnv(‘usr’, ‘test’, ‘3’); |
|
writing the complete IDS | Method of the ids object: def put(self, occurrence=0) Example: IdsNs::IDS imas_entry(12,1,0,0); imas_entry._pf_active.ids_properties.comment_of = "Test data"; imas_entry._pf_active.put(); for occurrence 0 imas_entry._pf_active.put(n); for occurrence n | Method of the ids object: def put(self, occurrence=0) Example: IdsNs::IDS imas_entry(12,1,0,0); imas_entry._pf_active.ids_properties.comment_of = "Test data"; imas_entry._pf_active.put(); for occurrence 0
| |
writing a single slice of an IDS | Method of the ids object: def putSlice(self, occurrence=0) Example: IdsNs::IDS imas_entry(12,1,0,0); imas_entry._actuator.putSlice();for occurrence 0 imas_entry._actuator.putSlice(n); for occurrence n | Method of the ids object: def putSlice(self, occurrence=0) Example: IdsNs::IDS imas_entry(12,1,0,0); imas_entry._actuator.putSlice();for occurrence 0 imas_entry._actuator.putSlice(n); for occurrence n | |
writing only non-timed data of an IDS | Method of the ids object: def putNonTimed(self, occurrence=0) Example: IdsNs::IDS imas_entry(12,1,0,0); imas_entry._pf_active.putNonTimed(); for occurrence 0 imas_entry._pf_active.putNonTimed(n); for occurrence n | Deprecated | |
reading the complete IDS | Method of the ids def get(self, occurrence=0) Example: IdsNs::IDS ids(12,1,0,0); ids.open(); //Open the database ids._pf_active.get(); for occurrence 0 ids._pf_active.get(n); for occurrence n | Method of the ids def get(self, occurrence=0) Example: IdsNs::IDS ids(12,1,0,0); ids.open(); //Open the database ids._pf_active.get(); for occurrence 0 ids._pf_active.get(n); for occurrence n | |
reading a single slice of an IDS | Method of the ids def getSlice(self, occurrence=0, time_requested, interpolation_method) Example: IdsNs::IDS ids(12,1,0,0); ids._pf_active.getSlice(time_requested, interpolation_method); for occurrence 0 ids._pf_active.getSlice(n, time_requested, interpolation_method); for occurrence n | Method of the ids def getSlice(self, occurrence=0, time_requested, interpolation_method) Example: IdsNs::IDS ids(12,1,0,0); ids._pf_active.getSlice(time_requested, interpolation_method); for occurrence 0 ids._pf_active.getSlice(n, time_requested, interpolation_method); for occurrence n | CLOSEST_SAMPLE PREVIOUS_SAMPLE INTERPOLATION |
copying an IDS or one of its substructure | |||
flush an IDS from the cache to the disk | Not implemented yet |
Deprecated | |
flushing all IDSes from the cache to the disk | def flushAll(self): e.g. imas_entry.flushAll() |
Deprecated | |
disables only temporarily the memory cache | Not implemented yet |
Deprecated | |
discarding all IDSes | def discardAll(self): eg. imas_entry.discardAll() |
Deprecated | |
deallocating an IDS or one if its substructure | |||
checking if arrays of an IDS were allocated in C (by AL) | |||
deleting an IDS from a data-entry | Method remove of the ids e.g. ids._pf_active.remove() | Method remove of the ids e.g. ids._pf_active.remove() | |
closing a data-entry | Method close() of the imas_entry object. e.g. IdsNs::IDS imas_entry(12,1,0,0); imas_entry.close(); def close(self) e.g. imas_entry.close() | Method close() of the imas_entry object. e.g. IdsNs::IDS imas_entry(12,1,0,0); imas_entry.close(); |
1.4. Python
Operation | AL3 | AL4 | Remarks |
---|---|---|---|
creating a data-entry (MDSplus backend) | A sequence of calls imas.ids(pulse, run) and create() on newly created object e.g. import imas ids_object = imas.ids(12,1) ids_object.create() sequence of calls imas.ids(pulse, run) and e.g. import imas ids_object = imas.ids(12,1) ids_object.create_env(‘usr’, ‘test’, ‘3’) | sequence of calls imas.ids(pulse, run) and e.g. import imas ids_object = imas.ids(12,1) ids_object.create_env(‘usr’, ‘test’, ‘3’) idx = imas_create_env( treename, pulse, run, 0, 0, UserName, DatabaseName,IMASMajorVersion ) |
|
opening a data-entry (MDSplus backend) | A sequence of calls imas.ids(pulse, run) and open() on newly created object e.g. import imas imas_entry = imas.ids(12,1,0,0) imas_entry.open() sequence of calls imas.ids(pulse, run) and open_env(UserName, DatabaseName, IMASMajorVersion) on newly created object e.g. import imas imas_entry = imas.ids(12,1) | A sequence of calls imas.ids(pulse, run) and open_env(UserName, DatabaseName, IMASMajorVersion) on newly created object e.g. import imas imas_entry = imas.ids(12,1) imas_entry.open_env(‘usr’, ‘test’, ‘3’) |
|
writing the complete IDS | Method of the imas_entry object: def put(self, occurrence=0) e.g. imas_entry.pf_active.put() for occurrence 0, or imas_entry.pf_active.put(occurrence) otherwise | Method of the imas_entry object: def put(self, occurrence=0) e.g.
| |
writing a single slice of an IDS | Method of the ids: def putSlice(self, occurrence=0): e.g. imas_entry.pf_active.putSlice() for occurrence 0, or imas_entry.pf_active.putSlice(occurrence) otherwise | Method of the ids: def putSlice(self, occurrence=0): e.g. imas_entry.pf_active.putSlice() for occurrence 0, or imas_entry.pf_active.putSlice(occurrence) otherwise | |
writing only non-timed data of an IDS | Method of the ids: def putNonTimed(self, occurrence=0): e.g. imas_entry.pf_active.putNonTimed() for occurrence 0, or imas_entry.pf_active.putNonTimed(occurrence) otherwise | Deprecated | |
reading the complete IDS | def get(self, occurrence=0): e.g. imas_entry.pf_active.get() for occurrence 0, or imas_entry.pf_active.get(occurrence) otherwise | def get(self, occurrence=0): e.g. imas_entry.pf_active.get() for occurrence 0, or imas_entry.pf_active.get(occurrence) otherwise | |
reading a single slice of an IDS | Method of the ids: def getSlice(self, time_requested, interpolation_method, occurrence=0): e.g. imas_entry.pf_active.getSlice(0.1, 2) for occurrence 0, or imas_entry.pf_active.getSlice(0.1, 2, occurrence) otherwise | Method of the ids: def getSlice(self, time_requested, interpolation_method, occurrence=0): e.g. imas_entry.pf_active.getSlice(0.1, 2) for occurrence 0, or imas_entry.pf_active.getSlice(0.1, 2, occurrence) otherwise | imasdef.py: UNDEFINED_INTERP CLOSEST_ INTERP PREVIOUS_INTERP LINEAR_INTERP |
copying an IDS or one of its substructure | |||
flush an IDS from the cache to the disk | Method of the ids: def flushCache(self, IDSOccurrence=0): e.g. 25 imas_entry.pf_active.flushCache() |
Deprecated | |
flushing all IDSes from the cache to the disk | def flushMemCache(self): e.g. imas_entry.flushMemCache() |
Deprecated | |
disables temporarily the memory cache | Method of the ids: def discard(self, IDSOccurrence=0): e.g. 26 imas_entry.pf_active.discard() |
Deprecated | |
discarding all IDSes | def discardMemCache(self): eg. imas_entry.discardMemCache() |
Deprecated | |
deallocating an IDS or one if its substructure | N/A | N/A | |
deleting an IDS from a data-entry | Method of the ids: def deleteData(self, occurrence=0): e.g. imas_entry.pf_active.deleteData() for occurrence 0, or imas_entry.pf_active.deletaData(occurrence) otherwise | Method of the ids: def deleteData(self, occurrence=0): e.g. imas_entry.pf_active.deleteData() for occurrence 0, or imas_entry.pf_active.deletaData(occurrence) otherwise | |
closing a data-entry | def close(self) e.g. imas_entry.close() | def close(self) e.g. imas_entry.close() |
1.5. Matlab
Operation | AL3 | AL4 | Remarks |
---|---|---|---|
creating a data-entry (MDSplus backend) | idx = imas_create(treename, pulse, run) idx = imas_create_env( treename, pulse, run, 0, 0, UserName, DatabaseName,IMASMajorVersion ) | idx = imas_create_env( treename, pulse, run, 0, 0, UserName, DatabaseName,IMASMajorVersion ) |
|
opening a data-entry (MDSplus backend) | idx = imas_create_env( treename, pulse, run, 0, 0, idx = imas_create(treename, pulse, run)UserName, DatabaseName,IMASMajorVersion )
| idx = imas_open_env( treename, pulse, run, UserName, DatabaseName, IMASMajorversion ) |
|
writing the complete IDS | ids_put(idx, IDSOccurrence, IDSVariable) | ids_put(idx, IDSOccurrence, IDSVariable) | |
writing a single slice of an IDS | ids_put_slice(idx, IDOccurrence, IDSVariable) | ids_put_slice(idx, IDOccurrence, IDSVariable) | |
writing only non-timed data of an IDS | ids_put_non_timed(idx, IDSOccurence, IDSVariable) | Deprecated | |
reading the complete IDS | IDSVariable = ids_get(expIdx, IDSOccurrence) | IDSVariable = ids_get(expIdx, IDSOccurrence) | |
reading a single slice of an IDS | IDSVariable = ids_get_slice(idx, IDSOccurrence, time_requested, interpolation_method) | IDSVariable = ids_get_slice(idx, IDSOccurrence, time_requested, interpolation_method) | |
copying an IDS or one of its substructure | |||
deleting an IDS from a data-entryn | Not implemented yet | Not implemented yet | |
closing a data-entry | [ cr ] = imas_close(idx) | [ cr ] = imas_close(idx) |
1.6. Java
Operation | AL3 | AL4 | Remarks |
---|---|---|---|
creating a data-entry (MDSplus backend) | imas.create(treename, pulse, run, 0, 0), for example : idx = imas.create(“ids” , 12, 1, 0, 0) ; imas.createEnv (treename, pulse, run, 0, 0, UserName, DatabaseName, IMASMajorVersion), for example : idx = imas.createEnv(“ids”, 12, 1, 0, 0, “user”, “test”, “3”) ; | imas.createEnv (treename, pulse, run, 0, 0, UserName, DatabaseName, IMASMajorVersion), for example : idx = imas.createEnv(“ids”, 12, 1, 0, 0, “user”, “test”, “3”) ; |
|
opening a data-entry (MDSplus backend) | Create IMAS entry object IdsNs::IDS imas_entry(pulse,run,0,0); then open() on the newly created object e.g. IdsNs::IDS imas_entry(12,1,0,0); imas_entry.open(); Create IDS object IdsNs::IDS imas_entry(pulse,run,0,0); then openEnv(UserName, DatabaseName, IMASMajorVersion) on the newly created object e.g. IdsNs::IDS imas_entry(12,1,0,0); imas_entry.openEnv(‘usr’, ‘test’, ‘3’); | imas.openEnv(treename, pulse, run, UserName, DatabaseName, IMASVersion), for example : idx = imas.openEnv(“ ids”, 12, 1, “user”, “test”, “3”) ; |
|
writing the complete IDS | Method of the ids: put (idx, IDSOccurrence, IDSVariable), for example : imas.equilibrium equilibrium = new imas.equilibrium(); equilibrium.put(idx, “equilibrium”, equilibrium) ; | Method of the ids: put (idx, IDSOccurrence, IDSVariable), for example : imas.equilibrium equilibrium = new imas.equilibrium();
| |
writing a single slice of an IDS | Method of the ids: putSlice (idx, IDSOccurrence, IDSVariable), for example : imas.equilibrium equilibrium = new imas.equilibrium(); equilibrium. putSlice(idx, “equilibrium”, equilibrium) ; | Method of the ids: putSlice (idx, IDSOccurrence, IDSVariable), for example : imas.equilibrium equilibrium = new imas.equilibrium(); equilibrium. putSlice(idx, “equilibrium”, equilibrium) ; | |
writing only non-timed data of an IDS | Method of the ids: putNonTimed (idx, IDSOccurrence, IDSVariable), for example : imas.equilibrium equilibrium = new imas.equilibrium(); equilibrium.putNonTimed(idx, “equilibrium”, equilibrium) ; | Deprecated | |
reading the complete IDS | Method of the ids: get (idx, IDSOccurrence, IDSVariable), for example : imas.equilibrium equilibrium = new imas.equilibrium(); equilibrium. get(idx, “equilibrium”, equilibrium) ; | Method of the ids: get (idx, IDSOccurrence, IDSVariable), for example : imas.equilibrium equilibrium = new imas.equilibrium(); equilibrium. get(idx, “equilibrium”, equilibrium) ; | |
reading a single slice of an IDS | Method of the ids: getSlice (idx, IDSOccurrence, IDSVariable, double time_requested, int interpolation_method), for example : imas.equilibrium equilibrium = new imas.equilibrium(); equilibrium.getSlice(idx, “equilibrium”, equilibrium, 0.1, 2) ; | Method of the ids: getSlice (idx, IDSOccurrence, IDSVariable, double time_requested, int interpolation_method), for example : imas.equilibrium equilibrium = new imas.equilibrium(); equilibrium.getSlice(idx, “equilibrium”, equilibrium, 0.1, 2) ; | |
copying an IDS or one of its substructure | |||
flushing all IDSes from the cache to the disk | imas.flush (idx, IDSOccurrence), for example : imas.flush (idx, “equilibrium/1”) ; |
Deprecated | |
flushing all IDSes from the cache to the disk | imas.flushAll(idx); |
Deprecated | |
disables temporarily the memory cache | imas.discard (idx, IDSOccurrence), for example : imas.discard (idx, “equilibrium”) ; |
Deprecated | |
discarding all IDSes | imas.discardAll (idx) ; |
Deprecated | |
deleting an IDS from a data-entry | Method of the ids: delete (idx, IDSOccurrence, IDSVariable), for example : imas.equilibrium equilibrium = new imas.equilibrium(); equilibrium.delete(idx, “equilibrium”, equilibrium) ; | Method of the ids: delete (idx, IDSOccurrence, IDSVariable), for example : imas.equilibrium equilibrium = new imas.equilibrium(); equilibrium.delete(idx, “equilibrium”, equilibrium) ; | |
closing a data-entry | imas.close(idx) ; | imas.close(idx) ; |
1.7. Lowlevel (C)