1. Issues

1.1. General

To access every machine I have to use separate configuration:

  • There is no single plugin installation that can handle all machines
  • There is no common installation of AL that works fine with all plugins

1.2. Runtime errors:

  • Core dumped when I try to access MAST using latest AL version
  • Software (Plugin? AL?) allocates an enormous amount of memory (about 200GB!), then killed by system (?) if I try to use Ludovic's plugin to access JET/MAST

1.3. IMAS based Docker 

  • IMAS based Docker is designed such way it supports one DD/AL release at a time
  • At the moment, handling all UDA sources requires multiple releases of DD/AL
  • We can't provide UDA support from within container as long as we don't have one, unified, UDA implementation at client side

1.4. Catalog QT

  • Update Process (responsible for reading pulse files) should be able to read data from a variety of sources (MDSPlus files, UDA)
  • At the moment, we can't do that by implementing a single, unified approach (each UDA source requires different settings)
  • This is not a blocker as we have decoupled import process such way it can handle individual sources separately. On a long run, this is probably something we want to avoid.

1.5. Users experience

  • At the moment, using UDA requires constant changing of the environment
  • Before using given plugin, user has to unload and load whole environment from the scratch
  • It is not possible to use two, different UDA sources at the same time

1.6. Content related issues

  • for some data we have -Inf/+Inf values inside pulse files
# To reproduce the case (at GW)

# Setup the environment

> module purge
> module load cineca
> module load imasenv/3.25.0/rc
> unsetenv UDA_PLUGIN
> setenv UDA_HOST rca.fusion.org.uk
> setenv UDA_PORT 56565

# Run the code

----- 8< --- CUT HERE --- 8< -----

import imas
import sys

def fun(shot, run, machine):

  mast=imas.ids(int(shot),int(run))
  mast.open_public(machine)
  mast.summary.get()

  print(mast.summary.global_quantities.h_98.value)

  time = mast.summary.time
  print('TIME: ', time)

  timeMode = mast.summary.ids_properties.homogeneous_time
  print('TIME MODE: ', timeMode)

if __name__ == "__main__":
  fun(sys.argv[1], sys.argv[2], sys.argv[3])

----- 8< --- CUT HERE --- 8< -----

> python mast.py 29650 0 MAST

# Results contain invalid data for signal: summary/global_quantities/h_98/value

IMAS_UDA::close()
[ -1.         -10.40032033 -19.80064066 -23.93528776 -14.78574923
  -3.58152153   0.08952234   0.12241262  -0.96586602  -1.
  -1.          -1.                  inf          nan   0.13058508
   0.10175894   0.09920737   0.12577316          inf          nan
          nan   0.11715577   0.08802964   0.07561934   0.10361142
   0.12758749   0.13078339   0.13665145   0.12083628   0.09869382
   0.11041405   0.246712     0.18799812   0.13322038   0.1721098
   0.31245901   0.20288732   0.27788885   0.79129496          inf
          nan          nan          nan   0.17371698   0.14705401
   0.16916175   0.8597472           inf          nan          nan
          nan          nan          nan          nan          nan
          nan          nan   0.13311852   0.13811199   0.14290544
   0.16826513   0.16323282   0.15154608   0.19396454   0.42245164
   0.66507736          inf          nan          nan          nan
   0.32031341   0.12473773   0.14420576   0.15714856   0.31664884
          nan          nan          nan   0.15312483   0.14305086
   0.14940249   0.14062285   0.23783689          nan          nan
   0.18147515   0.18369311   0.25258288   0.18228691   0.15957083
          inf          nan   0.13826879   0.17455354   0.28257633
   0.25866124   0.13272721          inf          nan  -1.        ]
TIME:  [0.01       0.01560606 0.02121212 0.02681818 0.03242424 0.0380303
 0.04363636 0.04924242 0.05484848 0.06045455 0.06606061 0.07166667
 0.07727273 0.08287879 0.08848485 0.09409091 0.09969697 0.10530303
 0.11090909 0.11651515 0.12212121 0.12772727 0.13333333 0.13893939
 0.14454545 0.15015151 0.15575757 0.16136364 0.1669697  0.17257576
 0.17818182 0.18378788 0.18939394 0.195      0.20060606 0.20621212
 0.21181818 0.21742424 0.2230303  0.22863636 0.23424242 0.23984848
 0.24545454 0.2510606  0.25666667 0.26227273 0.26787879 0.27348485
 0.27909091 0.28469697 0.29030303 0.29590909 0.30151515 0.30712121
 0.31272727 0.31833333 0.32393939 0.32954545 0.33515151 0.34075757
 0.34636363 0.3519697  0.35757576 0.36318182 0.36878788 0.37439394
 0.38       0.38560606 0.39121212 0.39681818 0.40242424 0.4080303
 0.41363636 0.41924242 0.42484848 0.43045454 0.4360606  0.44166666
 0.44727273 0.45287879 0.45848485 0.46409091 0.46969697 0.47530303
 0.48090909 0.48651515 0.49212121 0.49772727 0.50333333 0.50893939
 0.51454545 0.52015151 0.52575757 0.53136363 0.53696969 0.54257576
 0.54818182 0.55378788 0.55939394 0.565     ]
TIME MODE:  1


2. Current configurations

2.1. WEST

  • AL: IMAS/3.28.1/AL/develop
  • plugin: Ludovic's private one

2.2. JET

  • AL: IMAS/3.28.1/AL/develop
  • plugin: Stephen's private one

2.3. MAST:

  • AL: IMAS/3.25.0/AL/4.4.0
  •  plugin: a common one uda/2.2.5 

3. Things to be done

3.1. Integration of plug-ins

We need a single version AL+plugin that handles all machines. Could you kindly integrate your codes, check it against ALL machines and provide single, working installation?

3.2. Redesign of setting plugins 

A way of setting-up plugin should be redesign. Now (different!) system variables should be set to make it working. It prevents reading from different machines within one process (!)  IMHO UDA Backend based on a machine name should point to a correct plugin, without using system variables.

  • No labels