Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.


Info

The purpose of this documentation is to make users aware of how to download, compile and run server side components of Catalogue QT.

Table of contents

Table of Contents

What are we going to do?

This section of documentation explains how to install server part components of Catalogue QT. I am discussing here:

  • MySQL installation,
  • Spring Boot installation (execution of Spring Boot based application),
  • Exposing RESTful Web Services.

Schema of components used by RESTful WS serverImage Added

Starting WS API server from source codes

Catalogue QT Web Services are hosted using Spring-Boot framework. In order to start server you have to clone the repository with source code. Source

Executing Spring-Boot application (WS API)

...

code of all the components can be downloaded from the code repository (Catalog QT2).

Code Block
> git clone https://
YOUR_USER_NAME_AT_GATEWAY@gforge6
gitlab.eufus.
eu/git
psnc.pl/simulation_catalogue/catalog_qt_2.git


Info

If you don't have access yet, make sure to request for the access.

Requirements

Catalog Catalogue QT ver. 2 requires following components to be installed on the machine where it will be started

Code Block
Java         - openjdk 11.0.2 - you can use JDK from Oracle as well
Apache Maven - 3.6.3          - you can use Apache Maven distributed with project as well
mysql        - 8.0.20         - you can use MariaDB instead (we haven't tested it thoug)

Creating database

Catalog QT ver. 2 requires access to database that will store all the imported data. You have to create this database. Schema for creating database is located inside project structure: sql/schema/catalog_qt_db.sql.

...

Code Block
# WARNING! This line will drop schema: itm_catalog_qt and create it from the scratch. Make sure you are not destroying things.!

> mysql -u root -p < sql/schema/a_001_catalog_qt_db.sql
> mysql -u root -p < sql/schema/d_001_update_variables.sql

This script will create two users: itm_catalog_ro/itm_catalog_ro and user itm_catalog_rw/itm_catalog_rw with read-only and read-write privileges, respectively.

...

It's a good idea to check whether you can access your newly created database. Make sure you can successfully connect there.

Code Block
> mysql -u itm_catalog_rw -p -h localhost itm_catalog_qt
Enter password:
...
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 6318
Server version: 8.0.20 MySQL Community Server - GPL

...
...

mysql> show tables;
+
--
--------------------------+
| Tables_in_itm_catalog_
test
qt |
+
--
--------------------------+
| annotation               
|
| catalog_parameters
       |
| entry
                    |
| entry_data               |
| entry_replacement        |
| experiment
               |
| 
outcome
filter                   |
| 
reference
outcome                  |
| 
replace
reference                |
| request         
| |
 
request
        |
| request_status           |
| tag                      
|
| tag_entry
                |
| user                     |
| user_filter              |
| variable                 |
| variable_type            |
+--------------------------
--
+
13
17 rows in set (0.00 sec)


mysql>

Starting WS API server


Info

In case you have issues while accessing your MySQL installation, please consult your sys admin to resolve the issue.

 Starting Spring Web Services

To prepare Catalog QT environment run compile.sh script - it will create server application and basic components. 

Warning
titleJava based bridge to IMAS

Note that Catalogue QT Server requires IMAS based components. You have to import IMAS based JAR file into your local Maven repository. It's not something that will make you suffer. All you have to do is to run one command, and you have to make sure you have imas.jar file.

In order to run Catalogue QT ver. 2 Web Services In order to run WS API server you have to start Spring-Boot application. Make sure to go inside directory: server/catalog-ws-server and execute > java -jar target/catalog-ws -server-1.1.0-SNAPSHOT.jar

Code Block
> cd wsserver/catalog-ws-server/target/
> mvn spring-boot:runjava -jar target/catalog-ws-server-1.1.0-SNAPSHOT.jar
[INFO] Scanning for projects...
[INFO]
[INFO] ---------------------< pl.psnc.catalog:catalog-ws >---------------------
[INFO] Building catalog-ws 01.1.0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
...
...
 .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v2.3.0.RELEASE)

2020-06-05 13:21:31.533  INFO 17919 --- [           main] p.p.c.catalogws.CatalogWsApplication     : Starting CatalogWsApplication on catalog.eufus.eu with PID 17919
...
...

Running tests

You can run unit tests by changing directory to: ws/catalog-ws and running


Info

Note that you can use different port number for `Spring Boot` application

Code Block
> SERVER_PORT=8081 mvn spring-boot:run


 Catalogue QT Docker - Docker Compose Installation


This container is desined to simplify installation of Catalogue QT and it's components.
Instead of installing it on `IMAS` compatible platform you can use it on virtually any machine.


Known limitations

Note that this container should be used only for research purposes.


Info

Our codes and detailed documentation are open-source and you can find them in here: https://gitlab.eufus.psnc.pl/simulation_catalogue/catalogue_qt_docker


Starting Catalogue Update Process

Catalogue Update Process serves the purpose of reading data (MDSPlus pulse files, UDA, etc.) and populating Catalogue QT with the content of summary IDS. Catalogue Update Process is the only component that requires access to input data and to IMAS infrastructure. This is related to the fact that data are read from the structures created and maintained by IMAS based components.

Requirements

Catalogue QT Update Process requires following components to be installed on target machine:

Code Block
Java - openjdk 11.0.2 - you can use JDK from Oracle as well
Apache Maven - 3.6.3 - you can use Apache Maven distributed with project as well

Catalogue QT Update Process

Installation of Catalog QT CLI based client

I assume that you have already cloned the source code repository. In that case, all you have to do follows:

Warning
titleJava based bridge to IMAS

Note that Catalogue QT Update Process requires IMAS based components. You have to import IMAS based JAR file into your local Maven repository. It's not something that will make you suffer. All you have to do is to run one command, and you have to make sure you have imas.jar file.


Code Block
> cd catalog_qt_2/client/catalog-ws-client

Make sure to import imas.jar into your local Maven repository. All you have to do is to run following command

Code Block
> mvn org.apache.maven.plugins:maven-install-plugin:3.0.0-M1:install-file  \
    -Dfile=${LOCATION_OF_YOUR_IMAS_JAR}/imas.jar \
    -DgroupId=imas -DartifactId=imas \
    -Dversion=1.0.0-SNAPSHOT -Dpackaging=jar \
    -DlocalRepositoryPath=`pwd`/local-maven-repo

Once imas.jar is in place you can build and run Catalog QT CLI based client.

Code Block
> mvn install -DskipTests
> java -jar ./target/catalogAPI.jar -help

You should see the help screen with available options

Info
titleCatalog QT WS location

For the purpose of this documentation I assume that Catalog QT WS server is installed on localhost:8080.

Starting Catalogue QT Update

Warning

Please note that Update Process requires IMAS environment to work properly. If you don't have your IMAS environment set, it will complain

Code Block
> java -jar ./target/catalogAPI.jar  -startUpdateProcess --url http://localhost:8080
IMAS library not set up in the environment. (IMAS_PREFIX missing)

The easiest way to use IMAS environment is to run Update Process on the machine where IMAS environment is set. For example - Gateway.

Once you have everything in place you can start update process

Code Block
> java -jar ./target/catalogAPI.jar -startUpdateProcess --url http://localhost:8080
[main] INFO pl.psnc.catalog.client.cli.commands.StartUpdateProcess - Getting list of requests for processing from Catalog.
[main] INFO pl.psnc.catalog.client.cli.commands.StartUpdateProcess - Getting list of requests for processing from Catalog.
[main] INFO pl.psnc.catalog.client.cli.commands.StartUpdateProcess - Getting list of requests for processing from Catalog.
...
...

Starting Catalogue Update Process at Gateway - fast track

Info
titleDedicated WS machine

At Gateway there is a dedicated machine that hosts Catalogue QT Web Services - http://catalog:8080


The easiest way to get Catalogue QT Update Process running is to run it at Gateway. Here are the steps to run it

Code Block
# You need IMAS environment with Open JDK 11

> module purge
> module load cineca
> module load imasenv
> module unload itm-java
> module load openjdk

# You have to clone source repository (you need to request access if you haven't done so)
# https://gitlab.eufus.psnc.pl/simulation_catalogue/catalog_qt_2.git- and "Request to join project"

> git clone https://gitlab.eufus.psnc.pl/simulation_catalogue/catalog_qt_2.git
Cloning into 'catalog_qt_2'... 
Username for 'https://gitlab.eufus.psnc.pl': g2afilip
Password for 'https://g2afilip@gitlab.eufus.psnc.pl': 

> cd catalog_qt_2/client/catalog-ws-client/
> mvn org.apache.maven.plugins:maven-install-plugin:3.0.0-M1:install-file  \
    -Dfile=${IMAS_PREFIX}/jar/imas.jar \
    -DgroupId=imas -DartifactId=imas \
    -Dversion=1.0.0-SNAPSHOT -Dpackaging=jar \
    -DlocalRepositoryPath=`pwd`/local-maven-repo
> mvn install -DskipTests

# You are ready to run Update Process

> java -jar ./target/catalogAPI.jar -startUpdateProcess --url http://catalog:8080
[main] INFO pl.psnc.catalog.client.cli.commands.StartUpdateProcess - Getting list of requests for processing from Catalog.
...
...
Code Block
> mvn test