Info |
---|
This page was intended to collect information regarding evaluation of various technologies, libraries, database design and other activities related to initial architecture of the system. |
Table of Contents |
---|
Current state of tool
...
At the moment there are two, separate schemas for MySLQ based databases. First one, responsible for storing summary IDS data
Second one, responsible for storing information related to multiple databases
MariaDB data sharding
Main problems of the existing solution
...
Integration with IMAS Docker image
We plan to insert the environment of Catalog QT inside the IMAS Docker image.
This will allow you to automate the work of the tool, and allow any user to run Catalog QT on any machine that uses Docker.
All components will be inside Docker image: UI, API, and prepared but empty MySQL dataset. The user will be able to fill and manage freely.
IMAS Docker
IMAS Docker is a basis for Catalog QT Docker. It provides set of most basic components required to run IMAS based codes. You can find description of IMAS Docker at following location: WFMS:IMAS @Docker
There is a dedicated repository with samples you can use, for starters: https:
//
github.com/tzok/imas-hello-world.git.
Catalog QT Docker
Catalog QT Docker is built on top of IMAS Docker. In addition to IMAS components it introduces additional elements relevant to Catalog QT itself:
- MySQL database
- Java environment
- Spring Based web services
- TBD: Dashboard
This way, it provides all the components in one place. It is, however, possible to detach all the elements and run them separately. This would require manual intervention and customised, yet still possible.
Catalog QT Docker can be found at following location: https://github.com/mkowsiak/catalogue_qt_docker
Scenarios with Docker based installation
SOAP removal - time for JSON?
...
Current trends show that data formats such as JSON are more practical and allow the exchange of data with other systems in a more practical way.
If we want to integrate several tools, this should be done when the data format changes to a more universal and user-friendly one.
Data Feeder refactoring
At the moment, data feeder is tightly coupled with client application (CLI based client for Catalog QT). This was a natural choice as we planned to use only IMAS based data sources. Over the course of the development, it turned out that different data sources should be taken into consideration, for example: CSV files, HDF5 files, ASCII based formats. These formats are not supported by IMAS. It means, they shouldn't require IMAS dependency at all. Client code (CLI) depends on IMAS library (imas.jar). We want to avoid this dependency for other clients. This is why Data Feeder has to be moved into common part of codes, new data structures are needed to transfer data between common and client libraries, IMASFeeder should be the only IMAS dependent component. What we also need is a complete separation of URIParsers. Each client should depend on it's own URIParser implementation. These parsers should be either implemented inside common part, or stay close to client code responsible for reading data.