...
Tip |
---|
Quote of the day: |
Info |
---|
In this tutorial
|
Table of Contents | |
---|---|
|
Warning |
---|
Warning Stop here for a moment. Make sure you have followed the configuration setup before proceeding any further! Configuration related tutorial is here -> Click me! <- |
...
06.1.1. Data Entry
- A Data Entry is a collection of potentially all IDS
- Multiple occurrences of a given IDS can co-exist, e.g. multiple equilibria calculated by different codes / assumptions
- A Data Entry is defined by:
- IMAS version
- User name
- Machine name
- Pulse number
- Run number
For the future versions of UAL physical implementation of back-ends will be fully transparent, however current version of UAL is coupled only with MDSPlus engine used for storing/ reading data locally.
This part of tutorial provides you some tricks useful in everyday work.
...
06.1.2. MDSPlus database
MDSplus is a set of software tools for data acquisition and storage and a methodology for management of complex scientific data (source).
...
Explanation of the tree structure:
Location |
---|
Description | |
---|---|
$HOME/public/ |
imasdb/<machine_name>/<data_version>/<MDSPlus_tree_number>/ids_<shot><run>.* example: ~/public/imasdb/test/3/0/ids_120001.* | |
<machine_name> example: test | defines the equipment used for acquiring the results; "test" - is most commonly used for artificial, test data |
<data_version> example: 3 | Major Data Dictionary version |
<MDSPlus_tree_number> example: 0 | Contains all the shot and runs for given data tree |
Shot number | Shot number is represented by first digits in database file ids_SSSSRRRR.*; there is no limit for number of digits used |
Run number | Run number is represented by last four digits in database file ids_SSSSRRRR.* |
ids_####.datafile, ids_####.tree, ids_####.characteristics | Contains data for given shot/run |
...
06.1.3. Accessing MDSPlus database data
...
06.1.3.1 Listing pulse files
Info | ||
---|---|---|
| ||
Usage: imasdbs [OPTIONS] [COMMAND] This program lists existing databases. Possible commands are: list <shot number>- list existing databases slices <shot number> <run number> - list existing databases, including number of timeslices and time range for time-dependent IDSes times <shot number> <run number> - list existing databases, including number of timeslices their time points for time-dependent IDSes tokamak - list existing tokamaks (with data versions) dataversion - list existing dataversions (with tokamaks) If the optional arguments shot number and run number are given, only databases with these numbers will be shown. If no command is given, the list command is performed. To see databases stored in the public database, use 'public' as the user name. Options: -h, --help show this help message and exit -u USER, --user=USER Show databases of specified user -t TOKAMAK, --tokamak=TOKAMAK Show only databases for specified tokamaks -v VERSION, --version=VERSION Show only databases for specified data version --backend=BACKEND Show databases written with given backend(s). Comma- separated list of backends (Currently supported: mdsplus, hdf5). By default all backends are shown. -c, --compact Compact/reduced output |
...
Code Block |
---|
shell> imasdbs -t test slices 109999 402 Tokamak: test Data version: 3 UAL Backend: mdsplus Shot 10 Run: 40 core_profiles: 25 slices (345.0 - 345.48) core_sources: 25 slices (345.0 - 345.48) core_transport: 25 slices (345.0 - 345.48) equilibrium: 25 slices (345.0 - 345.48) transport_solver_numerics: 25 slices (345.0 - 345.48) wall: 25 slices (345.0 - 345.48) |
...
Code Block |
---|
shell> imasdbs -u palakb Tokamak: test Data version: 3 UAL Backend: mdsplus Shot 1 Runs: 1 Shot 2 Runs: 3 666 777 999 Shot 10 Runs: 30 40 42 60 61 64 65 66 80 81 99 123 666 999 1234 Shot 12 Runs: 1 2 99 Shot 13 Runs: 1 |
...
06.1.3.2 Dumping pulse files
To list the content (all data) of an IDS, use idsdump
script
Code Block |
---|
shell> idsdump
Usage: idsdump <USER> <TOKAMAK> <VERSION> <SHOT> <RUN> <IDS>
or
idsdump <SHOT> <RUN> <IDS> |
Code Block |
---|
shell> idsdump $USER test 3 9999 2 |
Code Block |
[palakb@hpc-login4 ~]$ idsdump 13 1 equilibrium class equilibrium Attribute ids_properties class ids_properties Attribute comment: Attribute homogeneous_time: 1 Attribute source: Attribute provider: Attribute creation_date: [.......] Attribute code class code Attribute name: 12 34 56 78 90 Attribute commit: 12 34 56 78 90 Attribute version: 12 34 56 78 90 Attribute repository: 12 34 56 78 90 Attribute parameters: 12 34 56 78 90 Attribute output_flag [-819925519 678927020 358961885 263985221 -518535735 -656888240 885898039 -949201251 187087431 189678740 306846126 536940120 -842545485 -121858537 -867824798 103609281 -986039164 -761981263 -444948662 -178414734 91809633 -65221224 575637439 -526052305] Attribute time [ 0. 1. 2.] |
...
1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18.
19. 20. 21. 22. 23. 24.]
|
06.1.3.2 Dumping an IDS node
Getting a subset of an IDS enables reading only a node (and its descendants if the node is a structure), making the GET operation much faster. To retrieve only requested node one should call the script idsdumppath
.
Code Block |
---|
idsdumppath
Usage: idsdumppath <USER> <TOKAMAK> <VERSION> <SHOT> <RUN> <IDS> <DATA_PATH> |
Path syntax:
- The path to requested node(s) is separated by slashes (“/path/to/node(s)”).
- Nodes representing arrays must contain indexes (“/path/to/array(idx)/field”) or “Fortran style” indices (“path/to/array(x:y)/field”)
- Limitation: In case of nested arrays, it is not allowed to specify set of indices for AoS ancestors. Only given values of AoS ancestors indices are handled: (e.g. “field/with/ancestorAoS(x:y)/field/AoS(n :m)” is not managed)
Data query examples:
- “flux_loop(1)/flux/data(1:5)”
- “bpol_probe(2:3)/field/data”
- “loop(:)/current”
- “time(4:-1)”
- “profiles_1d(2)/grid/rho_tor_norm(2:4)”
Code Block |
---|
shell> idsdumppath $USER test 3 9999 2 equilibrium "code"
Type: <class 'imas_3_24_0_ual_4_2_0.equilibrium.code__structure'>
----------------------------------------------
----------------------------------------------
class code
Attribute name: 12 34 56 78 90
Attribute commit: 12 34 56 78 90
Attribute version: 12 34 56 78 90
Attribute repository: 12 34 56 78 90
Attribute parameters: 12 34 56 78 90
Attribute output_flag
[-819925519 678927020 358961885 263985221 -518535735 -656888240
885898039 -949201251 187087431 189678740 306846126 536940120
-842545485 -121858537 -867824798 103609281 -986039164 -761981263
-444948662 -178414734 91809633 -65221224 575637439 -526052305]
|
Code Block |
---|
shell> idsdumppath $USER test 3 9999 2 equilibrium "code/output_flag(0)"
Type: <class 'numpy.int32'>
----------------------------------------------
----------------------------------------------
-819925519
|
06.1.3.3 Copying database files directly
In case you know user name, machine name, shot number and run number, you can import users' database files copying them directly from the users' public directories. Database files are located inside:
...
Code Block |
---|
# change directory in your $HOME
cd $HOME/public/imasdb/test/3/0/
# copy data files (pay attention to *_dot_* at the end of command line!)
cp ~michalo/public/imasdb/test/3/0/ids_120002.* .
cp ~michalo/public/imasdb/test/3/0/ids_130003.* . |
...
. |