Quote of the day:
I love the smell of broken configuration in the morning ...
- Anonymous tutorial lecturer
| Table of Contents |
|---|
1.1. 06.1.1. Data Entry
- A Data Entry is a collection of potentially all IDS
- Multiple occurrences of a given IDS can co-exist, e.g. multiple equilibria calculated by different codes / assumptions
- A Data Entry is defined by:
- IMAS version
- User name
- Machine name
- Pulse number
- Run number
For the future versions of UAL physical implementation of back-ends will be fully transparent, however current version of UAL is coupled only with MDSPlus engine used for storing/ reading data locally.
This part of tutorial provides you some tricks useful in everyday work.
1.2. 06.1.2. MDSPlus database
MDSplus is a set of software tools for data acquisition and storage and a methodology for management of complex scientific data (source).
In order to benefit from UAL layer, scientific data must be stored in MDSPlus database. Before you will be able to use it, you have to create database's structure. This is fairly simple. All you have to do is to execute following commands:
imasdb <machine_name>
At this point, you can check your public directory and browse through the database structure.
cd $HOME/public/imasdb/
Explanation of the tree structure:
| Location | Description |
|---|---|
| $HOME/public/itmdb/<machine_name>/<data_version>/<MDSPlus_tree_number>/ids_<shot><run>.* example: ~/public/imasdb/test/3/0/ids_120001.* | |
| <machine_name> example: test | defines the equipment used for acquiring the results; "test" - is most commonly used for artificial, test data |
| <data_version> example: 3 | Major Data Dictionary version |
| <MDSPlus_tree_number> example: 0 | Contains all the shot and runs for given data tree |
| Shot number | Shot number is represented by first digits in database file ids_SSSSRRRR.*; there is no limit for number of digits used |
| Run number | Run number is represented by last four digits in database file ids_SSSSRRRR.* |
ids_####.datafile, ids_####.tree, ids_####.characteristics | Contains data for given shot/run |
1.3. 06.1.3. Accessing MDSPlus database data
1.3.1. 06.1.3.1 Listing pulse files
itmdbs command
Usage: imasdbs [OPTIONS] [COMMAND]
This program lists existing databases.
Possible commands are:
list <shot number>- list existing databases
slices <shot number> <run number> - list existing databases, including number of timeslices and time range for time-dependent IDSes
times <shot number> <run number> - list existing databases, including number of timeslices their time points for time-dependent IDSes
tokamak - list existing tokamaks (with data versions)
dataversion - list existing dataversions (with tokamaks)
If the optional arguments shot number and run number are given, only databases with these numbers will be shown.
If no command is given, the list command is performed.
To see databases stored in the public database, use 'public' as the user name.
Options:
-h, --help show this help message and exit
-u USER, --user=USER Show databases of specified user
-t TOKAMAK, --tokamak=TOKAMAK
Show only databases for specified tokamaks
-v VERSION, --version=VERSION
Show only databases for specified data version
--backend=BACKEND Show databases written with given backend(s). Comma-
separated list of backends (Currently supported:
mdsplus, hdf5). By default all backends are shown.
-c, --compact Compact/reduced output
shell> imasdbs -t test slices 9999 2
Tokamak: test
Data version: 3
UAL Backend: mdsplus
Shot 10
Run: 40
core_profiles: 25 slices (345.0 - 345.48)
core_sources: 25 slices (345.0 - 345.48)
core_transport: 25 slices (345.0 - 345.48)
equilibrium: 25 slices (345.0 - 345.48)
transport_solver_numerics: 25 slices (345.0 - 345.48)
wall: 25 slices (345.0 - 345.48)
shell> imasdbs -u palakb
Tokamak: test
Data version: 3
UAL Backend: mdsplus
Shot 1 Runs: 1
Shot 2 Runs: 3 666 777 999
Shot 10 Runs: 30 40 42 60 61 64 65 66 80 81 99 123 666 999 1234
Shot 12 Runs: 1 2 99
Shot 13 Runs: 1
1.3.2. 06.1.3.2 Dumping pulse files
To list the content (all data) of an IDS, use idsdump script
shell> idsdump Usage: idsdump <USER> <TOKAMAK> <VERSION> <SHOT> <RUN> <IDS>
shell> idsdump $USER test 3 9999 2 equilibrium
class equilibrium
Attribute ids_properties
class ids_properties
Attribute comment:
Attribute homogeneous_time: 1
Attribute source:
Attribute provider:
Attribute creation_date:
[.......]
Attribute code
class code
Attribute name: 12 34 56 78 90
Attribute commit: 12 34 56 78 90
Attribute version: 12 34 56 78 90
Attribute repository: 12 34 56 78 90
Attribute parameters: 12 34 56 78 90
Attribute output_flag
[-819925519 678927020 358961885 263985221 -518535735 -656888240
885898039 -949201251 187087431 189678740 306846126 536940120
-842545485 -121858537 -867824798 103609281 -986039164 -761981263
-444948662 -178414734 91809633 -65221224 575637439 -526052305]
Attribute time
[ 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18.
19. 20. 21. 22. 23. 24.]
1.3.3. 06.1.3.2 Dumping an IDS node
Getting a subset of an IDS enables reading only a node (and its descendants if the node is a structure), making the GET operation much faster. To retrieve only requested node one should call the script idsdumppath .
idsdumppath Usage: idsdumppath <USER> <TOKAMAK> <VERSION> <SHOT> <RUN> <IDS> <DATA_PATH>
Path syntax:
- The path to requested node(s) is separated by slashes (“/path/to/node(s)”).
- Nodes representing arrays must contain indexes (“/path/to/array(idx)/field”) or “Fortran style” indices (“path/to/array(x:y)/field”)
- Limitation: In case of nested arrays, it is not allowed to specify set of indices for AoS ancestors. Only given values of AoS ancestors indices are handled: (e.g. “field/with/ancestorAoS(x:y)/field/AoS(n :m)” is not managed)
Data query examples:
- “flux_loop(1)/flux/data(1:5)”
- “bpol_probe(2:3)/field/data”
- “loop(:)/current”
- “time(4:-1)”
- “profiles_1d(2)/grid/rho_tor_norm(2:4)”
shell> idsdumppath $USER test 3 9999 2 equilibrium "code" Type: <class 'imas_3_24_0_ual_4_2_0.equilibrium.code__structure'> ---------------------------------------------- ---------------------------------------------- class code Attribute name: 12 34 56 78 90 Attribute commit: 12 34 56 78 90 Attribute version: 12 34 56 78 90 Attribute repository: 12 34 56 78 90 Attribute parameters: 12 34 56 78 90 Attribute output_flag [-819925519 678927020 358961885 263985221 -518535735 -656888240 885898039 -949201251 187087431 189678740 306846126 536940120 -842545485 -121858537 -867824798 103609281 -986039164 -761981263 -444948662 -178414734 91809633 -65221224 575637439 -526052305]
shell> idsdumppath $USER test 3 9999 2 equilibrium "code/output_flag(0)" Type: <class 'numpy.int32'> ---------------------------------------------- ---------------------------------------------- -819925519
1.3.4. 06.1.3.3 Copying database files directly
In case you know user name, machine name, shot number and run number, you can import users' database files copying them directly from the users' public directories. Database files are located inside:
~$USERNAME/public/imasdb/$TOKAMAKNAME/$DATAVERSION/0/ids_SSSSRRRR.*
Take a look at the example below. We will copy data from user michalo, machine test, shot: 12 and run: 2
# change directory in your $HOME cd $HOME/public/imasdb/test/3/0/ # copy data files (pay attention to *_dot_* at the end of command line!) cp ~michalo/public/imasdb/test/3/0/ids_120002.* . cp ~michalo/public/imasdb/test/3/0/ids_130003.* .