Table of Contents |
---|
Info | ||
---|---|---|
| ||
Installation of Catalog QT requires multiple components to be installed and configured to be fully operational. This is due to the fact that numerous, external, components are required during build procedure and during runtime. All these elements are described below. |
Installation of MySQL/MariaDB
Before you proceed with installation of Catalog QT (CLI/UI) you have to make sure there is a working MySQL server and you have a connection from Web Server and machine where CLI commands will be executed.
Make sure you can connect to server like this:
Code Block |
---|
# -u ual_user_s - you can use different user name # -h itmmysql1 $HOST_NAME - this is Gateway based host, but you can have something else here # it depends on your target installation # itmcatalog itm_scheduler_db - database name - you can decide on something else # but, by default, name of catalog db is 'itmcatalogitm_scheduler_db' # # Note that result of the command below might be different for you, depending # on whether you use MySQL or MariDB. Please, make sure that client library # for mysql will have the same version as your server. # # Note that your client may be executed on a completely different machine # than server # > mysql -u ual_user_s -h itmmysql1 itmcatalog$HOST_NAME itm_scheduler_db Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 25031 Server version: 5.6.36-log MySQL Community Server (GPL) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> |
...
Warning |
---|
one account with root privileges - you will have to create db schema one account with normal user privileges - this user must have access to all tables inside DB |
Installation of CLI based components
Installation of gSOAP 2.
...
8
In order to install CLI components you need to install and configure gSOAP 2.7 8 library. Please follow instructions here, and make sure gSOAP library is available in your system: gsoapgsoap_2.78.1370.zip
It's a good idea to provide separate module for gSOAP library that sets some basic locations
Code Block |
---|
module-whatis gsoap <http://http://gsoap2.sourceforge.net/> setenv GSOAP put_the_location_of_gsoap_here # e.g. /some_dir/libs/gsoap/2.78 prepend-path PATH put_location_of_bin_dir_here # e.g. /some_dir/libs/gsoap/2.78/bin prepend-path PKG_CONFIG_PATH put_location_of_pkg_config_here # e.g. /some_dir/libs/gsoap/2.78/lib/pkgconfig |
It's good to create gsoap.pc file - it can be used later, during compilation of the codes
Code Block |
---|
prefix=/some_dir/libs/gsoap/2.7
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include
Name: GSoap
Description: SOAP C Web Services
Version: 2.7
Libs: -L${libdir} -lgsoap
Cflags: -DWITH_DOM -I${includedir} |
Installation of IMAS UAL
Current version of Catalog QT requires at least imasIMAS/3.2023.02/ualAL/34.81.35
. Make sure to have IMAS
properly properly installed and configured at your target host. Please note that this manual describes steps for building codes based on branch - R2-newLL
it is assumed that codes will use New Low Level
features.
Installation of CLI tools
Checking out repository
Make sure you can access repository: https://gforge6.eufus.eu/svn/catalog_qt, before proceeding any further.
...
Code Block |
---|
. |-- C4gsoap - gSOAP communication layer in C++ |-- catalogScheduler - command line for data submission (called by user) |-- catalogTools - helper classes shared between projects |-- catalogUpdateProcess - process that actually performs data collection (executed on server machine) |-- dbAPI - API for database calls - shared between projects |-- docs - documentation |-- examples - sample files |-- modules - module for the project (used by users) |-- resources - place, where gSOAP layer is generated |-- scripts - helper scripts (store_data.sh, store_machine.sh) `-- src - UI - whole server part (UI); described later in this specification |
Warning | ||
---|---|---|
| ||
Since now, I am referring to sources inside: https://gforge6.eufus.eu/svn/catalog_qt/branches/R2-newLL |
Info | ||
---|---|---|
In following section I assume that
I am using |
Creating DB schema
Catalog QT tool requires multiple databases to run properly.
- Scheduler DB - this database defines all the machine dependent libraries
- Machine DB - this database contains data from the pulse file for given machine (there might be many machine databases)
Creating scheduler DB schema
Before compiling the code, make sure you can create db schema and that you can access it as user. You can create DB schema following way
Code Block |
---|
> cd docs/dbschema
> mysql -h $HOST_NAME -u user_with_root_access itm_scheduler_db -p < drop_scheduler_user.sql
> mysql -h $HOST_NAME -u user_with_root_access itm_scheduler_db -p < create_scheduler_db.sql
# Make sure you can access itm_scheduler_db DB
> mysql -h $HOST_NAME -u $UAL_USER itm_scheduler_db -p
Enter password:
...
...
mysql> show tables;
+----------------------------+
| Tables_in_itm_scheduler_db |
+----------------------------+
| access_list |
| db_authorization |
| scheduler |
| token_list |
+----------------------------+ |
Creating machine dependent DB schema
For each machine (e.g.: pedestal
, west
, jettc
) you have to have separate database schema.
Code Block |
---|
> cd docs/dbschema
# this script will produce SQL script for creating machine
# dependent databases
> ./create_db.sh --db-user=ual_user_001 \
--db-name=jettc \
--host-name=$HOST_NAME \
--schema-name=itm_catalog_001 \
--ws-address=https://address.of.itmcatalog.installation/itmcatalog/api \
--storage=/this/is/the/location/where/all/users/can/write/to
> mysql -h $HOST_NAME -u user_with_root_access -p < drop_db_user_modified.sql
> mysql -h $HOST_NAME -u user_with_root_access -p < create_db_modified.sql
# Once database is created, make sure you can log in
> mysql -h $HOST_NAME -u ual_user_001 -p itm_catalog_001
Enter password:
...
...
mysql> show tables;
+---------------------------+
| Tables_in_itm_catalog_001 |
+---------------------------+
| annotations |
| catalog_parameters |
| entry |
| entry_data |
| intervals |
| machines |
| outcome |
| scheduler |
| users |
| variables |
+---------------------------+
10 rows in set (0.00 sec) |
Populating database with variables
Each machine DB has to be populated with variables that are supposed to be stored inside DB. At the moment, machine dependent DB has to store all the names of fields stored inside summary
IDS. This can be done following way
Code Block |
---|
> xsltproc ./getFieldsIMAS.xsl \
$IMAS_HOME/core/installer/src/$IMAS_VERSION/ual/$UAL_VERSION/xml/IDSDef.xml | \
grep "summary" > \
variables.sql
> mysql -h $HOST_NAME -u ual_user_001 itm_catalog_001 -p < variables.sql
# Make sure you can see variables inside DB
> mysql -h $HOST_NAME -u ual_user_001 itm_catalog_001 -p
Enter password:
...
...
mysql> select count(*) from variables;
+----------+
| count(*) |
+----------+
| 2511 |
+----------+
1 row in set (0.00 sec) |
Compilation of CLI tools
Warning | ||
---|---|---|
| ||
gSOAP library requires WSDL location used for compilation of the codes. It is not possible to proceed any further without server components. This is related to the fact that gSOAP library requires WSDL location, where SOAP based specification is stored. |
Make sure to change directory to branches/R2-newLL
.
Code Block |
---|
# Make sure to load IMAS environment - following command may be system dependent
> module load imasenv
> imasdb test
# For bash
> export CATALOG_QT_HOME=`pwd`
> export CATALOG_QT_STORAGE=/this/is/the/location/where/all/users/can/write/to
> export CATALOG_QT_DB_PROPERTIES=${CATALOG_QT_HOME}/dbAPI
# For tcsh
> setenv CATALOG_QT_HOME `pwd`
> setenv CATALOG_QT_STORAGE /this/is/the/location/where/all/users/can/write/to
> setenv CATALOG_QT_DB_PROPERTIES ${CATALOG_QT_HOME}/dbAPI
# setting target machine
#
# unfortunately, there are few places where machine dependent locations are used
# at the moment, two options are available:
#
# setenv MACHINE gateway
# setenv MACHINE jet
#
# it might be, we need yet another one, with new machine settings
# make sure to create and properly configure gSOAP module
# this way, you will be able to use pkg-config, etc.
> module load gsoap/2.8
> make |
Installation of UI - web client
Make sure you can access repository: https://gforge6.eufus.eu/svn/catalog_qt
Code Block |
---|
> svn co https://gforge6.eufus.eu/svn/catalog_qt |
The tool requires the following components to work:
- MySQL database management system
- Apache HTTP Server
- PHP server-side scripting language
Installation of server tools
The administrator should install the following packages on the server:
- mysql-server or mariadb-server
- php
- libapache2-mod-php
- apache2
Installation of web client
UI codes are available as a sub-project in the src directory
Code Block |
---|
src
|-- api_1_0 - API routines
|-- client - nusoap wrappers
|-- config.php - main configuration file
|-- inc - auxiliary scripts
|-- nusoap - nusoap library
|-- test.php - nusoap testing script
`-- ui2 - main UI scripts and web page |
To install the UI, copy the src folder to the apache directory (usually it is /var/www/html/, but it depends on the version and type of operating system)
It is best to add an alias in the httpd.conf file to access the page after a short link without entering the entire address.
After all, restart the apache server:
Code Block |
---|
> service apache2 restart |
Configuration of UI
the fastest way:
Because the codes are mainly prepared for operation on the GW machine, to complete the configuration, all the names of the GW server (http://portal.eufus.eu, https://portal.eufus.eu) should be replaced with the names of the new server.
These names should be located and be replaced in these files:
Code Block |
---|
./src/ui2/config.php
./src/config.php |
You can configure access to the mysql database in a ./src/ui2/config.php file (The data should match those given in the section Creating db schema and Installation of MySQL/MariaDB).
Also configuration of API databases access should be similarly adapted:
Code Block |
---|
./src/config_scheduler_db.php |
After all, restart the apache server:
Code Block |
---|
> service apache2 restart |
Access to databases by API
The latest version of the API allows communication with many databases through one unified interface.
Thanks to this, there is no need to create a separate API instance for each database on the server side.
The client can query one WS address to have access to all databases available on the server.
In the client's SOAP request header, the variable Qdb contains the database name.
The diagram below shows how this mechanism works:
Mermaid |
---|
sequenceDiagram
Client->>Catalog API: SOAP request [header with Qdb variable]
Catalog API->>Scheduler DB: ask for DB configuration
Scheduler DB->>Catalog API: send DB configuration
Catalog API->>Catalog API: switch to chosen DB
Catalog API->>Chosen DB: request/command to execute
Chosen DB->>Chosen DB: execute request/command
Chosen DB-->>Client: send feedback |
The API reads the Qdb variable from the header only if the server needs this information. Otherwise, server ignores client's header:
Mermaid |
---|
sequenceDiagram
Client->>Catalog API: SOAP request [header with Qdb variable]
Catalog API->>Scheduler DB: request/command to execute
Scheduler DB->>Scheduler DB: ignore header [Qdb variable]
Scheduler DB->>Scheduler DB: execute request/command
Scheduler DB-->>Client: send feedback
participant Other Databases |