Current state of tool

...

• Client (written in C/C++) - does not have to work on the machine on which the API and UI work. Allows user to add new data to the Catalog dataset.
• UI (written in PHP/javascript) - server part. It allows you to present the collected data. For now, it is not communicating remotely with the API. Both must be on the same machine.
• API (written in PHP) - server part. Handles client and UI SOAP requests. The middleman between them and MySQL.
• MySQL - serves as a source of the data.

MySQL database schemas

At the moment there are two, separate schemas for MySLQ based databases.

...

First one, responsible for storing summary IDS data

Database schema

MariaDB data sharding

Database shardingImage Removed

Main problems of the existing solution

...

Integration with IMAS Docker image

We plan to insert the environment of Catalog QT inside the IMAS Docker image.
This will allow you to automate the work of the tool, and allow any user to run Catalog QT on any machine that uses Docker.

IMAS Docker

IMAS Docker is a basis for Catalog QT Docker. It provides set of most basic components required to run IMAS based codes. You can find description of IMAS Docker at following location: WFMS:IMAS @Docker

There is a dedicated repository with samples you can use, for starters: https://github.com/tzok/imas-hello-world.git. 

Catalog QT Docker

Catalog QT Docker is built on top of IMAS Docker. In addition to IMAS components it introduces additional elements relevant to Catalog QT itself:

• MySQL database
• Java environment
• Spring Based web services
• TBD: Dashboard

This way, it provides all the components in one place. It is, however, possible to detach all the elements and run them separately. This would require manual intervention and customised, yet still possible.

Catalog QT Docker can be found at following location: https://github.com/mkowsiak/catalogue_qt_docker

Scenarios with Docker based installation

Image AddedAll components will be inside Docker image: UI, API, and prepared but empty MySQL dataset. The user will be able to fill and manage freely.

SOAP removal - time for JSON?

...

Current trends show that data formats such as JSON are more practical and allow the exchange of data with other systems in a more practical way.
If we want to integrate several tools, this should be done when the data format changes to a more universal and user-friendly one.

Data Feeder refactoring

At the moment, data feeder is tightly coupled with client application (CLI based client for Catalog QT). This was a natural choice as we planned to use only IMAS based data sources. Over the course of the development, it turned out that different data sources should be taken into consideration, for example: CSV files, HDF5 files, ASCII based formats. These formats are not supported by IMAS. It means, they shouldn't require IMAS dependency at all. Client code (CLI) depends on IMAS library (imas.jar). We want to avoid this dependency for other clients. This is why Data Feeder has to be moved into common part of codes, new data structures are needed to transfer data between common and client libraries, IMASFeeder should be the only IMAS dependent component. What we also need is a complete separation of URIParsers. Each client should depend on it's own URIParser implementation. These parsers should be either implemented inside common part, or stay close to client code responsible for reading data.

Image Added