...
- MySQL installation,
- Spring Boot installation (execution of Spring Boot based application),
- Exposing RESTful Web Services.
Starting
...
WS API server from source codes
Catalogue QT Web Services are hosted using Spring-Boot framework. In order to start server you have to clone the repository with source code. Source code of all the components can be downloaded from the code repository (Catalog QT2).
...
Info |
---|
In case you have issues while accessing your MySQL installation, please consult your sys admin to resolve the issue. |
...
Starting Spring Web Services
To prepare Catalog QT environment run compile.sh script - it will create server application and basic components.
...
Info | ||
---|---|---|
Note that you can use different port number for `Spring Boot` application
|
Startnig WS API Server from Docker Containers
Our codes are open-source and you can find them in here: https://github.com/mkopsnc/catalogue_qt_docker .
We have two docker containers that containes all of the above components.
- docker compose - the app is built of independent dockers that are connected to each other. According to the idea of microservices.
- single-container - mainly used by developers, all components are built on the basis of one Dockerfile, which creates one container.
Docker-compose installation
Catalogue QT Docker - Docker Compose Installation
This container is desined to simplify installation of Catalogue QT and it's components. Instead of installing it on `IMAS` compatible platform you can use it on virtually any machine.
Info |
---|
Our codes are open-source and you can find them in here: https://github.com/mkopsnc/catalogue_qt_docker |
Info |
---|
If you are installing our docker for the first time please go to the next section of this documentation. |
...
Code Block |
---|
> git clone https://github.com/mkopsnc/catalogue_qt_docker.git > cd docker-compose/build > ./build.sh > cd .. > ./run.sh -s notoken |
Prepare your work environment
In order to build this container, you will need access to few repositories. This container is based on:
...
Code Block |
---|
> git clone --single-branch master https://github.com/tzok/imas-watchdog.git |
Building container
In order to build and run container you have to do following:
Code Block |
---|
> cd docker-compose/build > ./build.sh |
Starting container
Catalogue QT 2 Docker can be run using multiple configurations. By default we provide following configurations
...
Code Block |
---|
> cd docker-compose # ./run.sh -s <configuration file suffix> e.g. > ./run.sh -s notoken |
To access our application please paste this urls in your browser:
- http://localhost:8080/swagger-ui.html to access Web Services via Swagger based UI.
- http://localhost.dashboard-ui.pl:9100/dashboard/ to access Dashboard-ReactJS
Configuration
Docker-compose Configuration
You can edit docker-compose._deployment_name_.yml
to change:
- The path where MySQL will store the data (default:
$(pwd)/db-data
) - The path where pulsefiles are stored on the host (default:
$(pwd)/imasdb
) - To map MySQL port to host port, so you can access the database from the container (by deafult no ports are exposed)
- To add custom configuration of Web Services:
application.properties
file
Additionally you can edit existing configuration, or create your own e.g docker-compose.myconf.yml
and run it!
Code Block |
---|
> ./run.sh -s myconf |
Catalog QT 2 Web Services Configuration
Moreover, in our catalog-ws-server
we have application.properties
file, which is a configuration for our Web Services in Springboot.
These Web Services are run inside Tomcat server.
Anatomy of application.properties file
Code Block |
---|
# location of database - typically, it will point at localhost, but # it's also possible to change location of MySQL server. # Docker based installation (docker-compose) will change it to db:3306 # In case of docker-compose based installation, MySQL is visible as another host # Note that you don't have to change anything spring.datasource.url=jdbc:mysql://localhost:3306/itm_catalog_qt?serverTimezone=UTC # Default user name and password for database connection. Note that this connection # will not work (by default) for external hosts. This is why we don't quite care about # user/pass - however, you can alter these and make sure they don't contain default values spring.datasource.username=itm_catalog_rw spring.datasource.password=itm_catalog_rw spring.jpa.properties.hibernate.jdbc.time_zone=UTC # In case of errors we want to embed error message as well (so we better know what went wrong) server.error.include-message=always # We definitely don't want to log SQL queries. However, if you want to see them, feel free # to enable this property spring.jpa.show-sql=false # We don't want to generate DB schema from bean classes spring.jpa.hibernate.ddl-auto=none # This is additional http handler, on another port # We need this one, in case we plan to use https # This is tricky :) # If server.ssl fields are set, this field defines https port # If server.ssl fields are not set, this field defines http port server.port=8080 # However, we need http port anyway (for some components). This is why we expose services on # http anyway. At the and we can end up with two different configurations # http (8080) and http (8081) - no certificates # https (8443) and http (8081) - certificates server.http.port=8081 server.http.interface=0.0.0.0 # ------- Keycloak settings ------- keycloak.enabled=true keycloak.realm = fair4fusion-docker-demo keycloak.auth-server-url=https://sso.apps.paas-dev.psnc.pl/ keycloak.resource= catalogqt-cli keycloak.realm-key= MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAjOCDGJsBi7rxVjf0RQb8pm0LAGsEKFcH7g7mKSqpFvp1uOypUeiYe5dwlwkXAXaYeYs0J70LB8E6mtVUcykbmp+XrqD1nn3yfPxlVLSg7iCvJqMUq8udsUbsyT3M/32/kssXurgY7rX5JhdtkYeAgq+9ifIjLQZhALg+FvEsX9C+D30WQDAChEljlReb+Y4UTz2aIqz9C+90bqG1ZIX4o3Dli1PZDosTNM444CwDTbrFrenctOTDtGPodo9k2jze8McZFAIrdUYi9mKD8v0frs8NUUW/TQj9h62swXdvVAfzYTd+R7aMRG0eXMV3rJc38DfsCsF7bkqSg0b4l8GcaQIDAQAB keycloak.bearer-only = true keycloak.public-client=true keycloak.principal-attribute=preferred_username spring.mvc.dispatch-options-request=true # ------- HTTPS settings -------- # If you plan to use HTTPS, make sure to uncomment this one # You have to make sure to generate and configure SSL certificate for your domain #server.ssl.key-store=file:///home/imas/cert/keystore.p12 #server.ssl.key-store-password=catalogqt #server.ssl.keyStoreType=PKCS12 #server.ssl.keyAlias=tomcat # ------- Bearer token authorization -------- # Should we check authorisation header or not. This feature toggle enables sort of "single user mode" # It's useful in case you don't have Keycloak and don't care about user roles. Once set to "false" # it will make Web Services behave as if there is only one user swagger-ui.authorization.header=true |
The default configuration is inside our project, but (before building) if you want to use a diffrent configuration (e.g enabling SSL certificates, enabling Keycloak) you can paste in folder /catalogue_qt_docker/docker-compose/build/files/server
another application.properties
file, which will have higher priority and would override existing file in source codes and then you can build and run our docker.
If you have already build container, and want to change Web Services configuration, you can do that without rebuilding docker!
All you need to do is to add application.properties
file to this folder docker-compose/volumes/server-properties
.
When the container is taken off, it will have the highest priority.
After changing the settings, it may be necessary to restart from scratch:
Code Block |
---|
> docker-compose rm > docker-compose up |
...
Congratulations! You have set up an SSL certificate!
Adding persistent storage
You can add persistent storage by setting it up inside docker-compose.yml
file
Code Block |
---|
services: db: volumes: - ./volumes/mysql:/var/lib/mysql |
...
It is not required to link ./volumes/mysql
location. In case you are using some other location for persistent data, feel free to use it instead.
Importing data from pulse file
Catalog QT Demonstrator allows to import MDSPlus based data automatically into SQL database. In order to do this you have to bind mount a volume. In a plain text it means that you have to tell Docker that you want to make your local filesystem to be available inside Docker container. Easiest way to do it is to create directory (or symbolic link) to a MDSPlus compatible local database.
First of all, make sure you have MDSPlus
like directory structure with pulse files. The easiest way to execute Docker container with sample data is to get sample data from box.psnc.pl
- these are completely artificially created data prepared by testing framework.
Code Block |
---|
> curl -s -o f4f_data.tar.gz \ https://box.psnc.pl/seafhttp/files/01953e73-8ad3-4277-be71-57b69c395355/f4f_data.tar.gz |
Make sure your directories structure looks like this:
Code Block |
---|
. `-- catalogue_qt_docker `-- docker-compose `-- volumes `-- imasdb |-- f4f | `-- 3 | |-- 0 | | |-- ids_11062020.characteristics | | |-- ids_11062020.datafile | | |-- ids_11062020.populate | | |-- ids_11062020.tree | | |-- ids_11062021.characteristics | | |-- ids_11062021.datafile | | |-- ids_11062021.populate | | `-- ids_11062021.tree | |-- 1 | |-- 2 | |-- 3 | |-- 4 | |-- 5 | |-- 6 | |-- 7 | |-- 8 | `-- 9 `-- script.sh |
Directory catalogue_qt_docker/docker-compose/volumes/imasdb
is automatically mounted inside Docker container. It means that anything you have put in it, will be visible inside Docker container whenever it is running. Once Docker is running you can schedule data population by creating file with *.populate
extension. You can do it following way. Inside directory with data execute script.sh
with the name of database you want to have populated.
Code Block |
---|
> cd catalogue_qt_docker/docker-compose/volumes/imasdb > ./script.sh f4f |
If anything goes wrong, please delete all the .populate
files by executing this command on linux:
Code Block |
---|
> find . -type f -name "*.populate" -delete |
Debugging in docker-compose
You can debug either all the Java based components, inside Docker container, or you can specify which one should be started in debug more. For debugging Java code inside Docker containers we are using JDWP
protocol, and by default we are using following ports
Info |
---|
|
In order to enable debbug mode you can either use predefined docker-compose.debug.yml
or enable debug mode for each service separatelly by adding sections inside your YAML file of choice.
Catalog-ws-server
To debug catalog-ws-server you need to add following lines to docker-compose.####.yml
in server
section
Code Block |
---|
server: volumes: - ./volumes/imasdb:/home/imas/public/imasdb ports: - 32889:32889 environment: - DEBUG_SPRING_BOOT=true |
Update process
To debug update-process you need to add following lines to docker-compose.####.yml
in updateprocess
section
Code Block |
---|
updateprocess: volumes: - ./volumes/imasdb:/home/imas/public/imasdb ports: - 32888:32888 environment: - DEBUG_UPDATE_PROCESS=true |
...
To debug imas-watchdog you need to add following lines to docker-compose.####.yml
in updateprocess
section
Code Block |
---|
watchdog: volumes: - ./volumes/imasdb:/home/imas/public/imasdb - ./volumes/fair4fusion-docker-demo:/docker-entrypoint-properties.d ports: - 32887:32887 environment: - DEBUG_IMAS_WATCHDOG=true |
...