...
At the moment there are two, separate schemas for MySLQ based databases. First one, responsible for storing summary IDS data
MariaDB data sharding
Main problems of the existing solution
...
Catalog QT Docker can be found at following location: https://github.com/mkowsiak/catalogue_qt_docker
Scenarios with Docker based installation
SOAP removal - time for JSON?
...
Current trends show that data formats such as JSON are more practical and allow the exchange of data with other systems in a more practical way.
If we want to integrate several tools, this should be done when the data format changes to a more universal and user-friendly one.
Data Feeder refactoring
At the moment, data feeder is tightly coupled with client application (CLI based client for Catalog QT). This was a natural choice as we planned to use only IMAS based data sources. Over the course of the development, it turned out that different data sources should be taken into consideration, for example: CSV files, HDF5 files, ASCII based formats. These formats are not supported by IMAS. It means, they shouldn't require IMAS dependency at all. Client code (CLI) depends on IMAS library (imas.jar). We want to avoid this dependency for other clients. This is why Data Feeder has to be moved into common part of codes, new data structures are needed to transfer data between common and client libraries, IMASFeeder should be the only IMAS dependent component. What we also need is a complete separation of URIParsers. Each client should depend on it's own URIParser implementation. These parsers should be either implemented inside common part, or stay close to client code responsible for reading data.
