Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents


Info
titleComponents

Installation of Catalog QT requires multiple components to be installed and configured to be fully operational. This is due to the fact that numerous, external, components are required during build procedure and during runtime. All these elements are described below.

Installation of MySQL/MariaDB

Before you proceed with installation of Catalog QT (CLI/UI) you have to make sure there is a working MySQL server and you have a connection from Web Server and machine where CLI commands will be executed.

Make sure you can connect to server like this:

Code Block
# -u ual_user  _s    - you can use different user name
# -h itmmysql1 $HOST_NAME    - this is Gateway based host, but you can have something else here
#                    it depends on your target installation
# itmcatalog       itm_scheduler_db - database name - you can decide on something else
#                    but, by default, name of catalog db is 'itmcatalogitm_scheduler_db'
#
# Note that result of the command below might be different for you, depending
# on whether you use MySQL or MariDB. Please, make sure that client library 
# for mysql will have the same version as your server.
#
# Note that your client may be executed on a completely different machine
# than server
#
 
> mysql -u ual_user_s -h itmmysql1 itmcatalog$HOST_NAME itm_scheduler_db
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 25031
Server version: 5.6.36-log MySQL Community Server (GPL)
Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql>
 

...

Warning

one account with root privileges - you will have to create db schema

one account with normal user privileges - this user must have access to all tables inside DB

Installation of CLI based components

Installation of gSOAP 2.

...

8

In order to install CLI components you need to install and configure gSOAP 2.7 8 library. Please follow instructions here, and make sure gSOAP library is available in your system:                          gsoapgsoap_2.78.1370.zip

It's a good idea to provide separate module for gSOAP library that sets some basic locations

Code Block
module-whatis    gsoap <http://http://gsoap2.sourceforge.net/>
setenv           GSOAP put_the_location_of_gsoap_here               # e.g. /some_dir/libs/gsoap/2.78
prepend-path     PATH put_location_of_bin_dir_here                  # e.g. /some_dir/libs/gsoap/2.78/bin
prepend-path     PKG_CONFIG_PATH put_location_of_pkg_config_here    # e.g. /some_dir/libs/gsoap/2.78/lib/pkgconfig

It's good to create gsoap.pc file - it can be used later, during compilation of the codes

Code Block
prefix=/some_dir/libs/gsoap/2.7
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include

Name: GSoap
Description: SOAP C Web Services
Version: 2.7


Libs: -L${libdir} -lgsoap
Cflags: -DWITH_DOM -I${includedir}

Installation of IMAS UAL

Installation of IMAS UAL

Current version of Catalog QT requires at least IMAS/3.23.2/AL/4.1.5 . Make sure to have IMAS properly Current version of Catalog QT requires at least imas/3.20.0/ual/3.8.3. Make sure to have IMAS properly installed and configured at your target host. Please note that this manual describes steps for building codes based on branch - R2-newLL it is assumed that codes will use New Low Level features.

Installation of CLI tools

Checking out repository

Make sure you can access repository: https://gforge6.eufus.eu/svn/catalog_qt, before proceeding any further.

...

Warning
titleCatalogQT version

Since now, I am referring to sources inside: https://gforge6.eufus.eu/svn/catalog_qt/branches/R1.0.imas

 

Creating db schema

Before compiling the code, make sure you can create db schema and that you can access it as user. You can create DB schema following way

R2-newLL


Info

In following section I assume that

Code Block
$HOST_NAME            - points to location of MySQL/MariaDB server
user_with_root_access - is a name of MySQL's user name with root privileges
$UAL_USER    
Code Block
> cd docs/dbschema > mysql -h $HOST_NAME -u $UAL_USER itmcatalog -p < create_db.sql   # You have to alter table where some config info is stored make sure to call   > cat alter_00.sql | \
         - is the name of the user who has access to itm_scheduler_db
                        (this user is create using create_scheduler_db.sql script)

I am using -p option while calling mysql - it means that you will be asked for password each time you want to log in to MySQL machine. Thus, it is not possible to use following samples in batch mode.

Creating DB schema

Catalog QT tool requires multiple databases to run properly.

  • Scheduler DB - this database defines all the machine dependent libraries
  • Machine DB - this database contains data from the pulse file for given machine (there might be many machine databases)

Creating scheduler DB schema

Before compiling the code, make sure you can create db schema and that you can access it as user. You can create DB schema following way

Code Block
> cd docs/dbschema
> mysql -h $HOST_NAME -u user_with_root_access itm_scheduler_db -p < drop_scheduler_user.sql
> mysql -h $HOST_NAME -u user_with_root_access itm_scheduler_db -p < create_scheduler_db.sql
 
# Make sure you can access itm_scheduler_db DB
sed 's|@FULL_PATH_FOR_STORAGE@|/this/is/the/location/where/all/users/can/write|' > \
      alter_00_modified.sql
    mysql -h itmmysql1 -u ual_user_s itmcatalog -p < alter_00_modified.sql
 
# /this/is/the/location/where/all/users/can/write - is a place on your machine (where users are calling
#                                                   scheduler application). Each user has to have write
#                                                   access there.
 
> mysql -h $HOST_NAME -u $UAL_USER itmcatalog -p < alter_00_modified.sql
 
 
# make sure to create list of variables that are stored inside DB
 
> xsltproc ./getFieldsIMAS.xsl \
      $IMAS_HOME/core/installer/src/$IMAS_VERSION/ual/$UAL_VERSION/xml/IDSDef.xml | \
      grep "summary" > \
      variables.sql
> mysql -h $HOST_NAME -u $UAL_USER itmcatalogitm_scheduler_db -p
Enter < variables.sql
 
# double check that all tables are in proper place
 
> mysql -h $HOST_NAME -u $UAL_USER itmcatalog -p
> ...
> mysql> show tables;
+password: 
...
...
mysql> show tables;
+----------------------------+
| Tables_in_itm_scheduler_itmcatalogdb |
+----------------------------+
| access_list  annotations          |
| catalog_parameters   |
| entrydb_authorization                |
| entry_data           |
| intervals            |
| machines             |
| outcome              |
| scheduler            |
| users                |
| variables            |
+----------------------+
10 rows in set (0.00 sec)

 

...

|
| scheduler                  |
| token_list                 |
+----------------------------+

Creating machine dependent DB schema

For each machine (e.g.: pedestalwestjettc) you have to have separate database schema.

Code Block
> cd docs/dbschema

# this script will produce SQL script for creating machine
# dependent databases

> ./create_db.sh --db-user=ual_user_001 \
                 --db-name=jettc \
                 --host-name=$HOST_NAME \
                 --schema-name=itm_catalog_001 \
                 --ws-address=https://address.of.itmcatalog.installation/itmcatalog/api \
                 --storage=/this/is/the/location/where/all/users/can/write/to

> mysql -h $HOST_NAME -u user_with_root_access -p < drop_db_user_modified.sql
> mysql -h $HOST_NAME -u user_with_root_access -p < create_db_modified.sql

# Once database is created, make sure you can log in

> mysql -h $HOST_NAME -u ual_user_001 -p itm_catalog_001
Enter password: 
...
...
mysql> show tables;
+---------------------------+
| Tables_in_itm_catalog_001 |
+---------------------------+
| annotations               |
| catalog_parameters        |
| entry                     |
| entry_data                |
| intervals                 |
| machines                  |
| outcome                   |
| scheduler                 |
| users                     |
| variables                 |
+---------------------------+
10 rows in set (0.00 sec)

Populating database with variables

Each machine DB has to be populated with variables that are supposed to be stored inside DB. At the moment, machine dependent DB has to store all the names of fields stored inside summary IDS. This can be done following way

Code Block

> xsltproc ./getFieldsIMAS.xsl \
      $IMAS_HOME/core/installer/src/$IMAS_VERSION/ual/$UAL_VERSION/xml/IDSDef.xml | \
      grep "summary" > \
      variables.sql

> mysql -h $HOST_NAME -u ual_user_001 itm_catalog_001 -p < variables.sql

# Make sure you can see variables inside DB

> mysql -h $HOST_NAME -u ual_user_001 itm_catalog_001 -p
Enter password: 
...
...
mysql> select count(*) from variables;
+----------+
| count(*) |
+----------+
|     2511 |
+----------+
1 row in set (0.00 sec)

Compilation of CLI tools

Warning
titlegSOAP and WSDL

gSOAP library requires WSDL location used for compilation of the codes. It is not possible to proceed any further without server components. This is related to the fact that gSOAP library requires WSDL location, where SOAP based specification is stored.

Make sure to change directory to branches/R2-newLL.

Code Block
# Make sure to load IMAS environment - following command may be system dependent
> module load imasenv
> imasdb test
 
# For bash
 
> export CATALOG_QT_HOME=`pwd`
> export CATALOG_QT_STORAGE=/this/is/the/location/where/all/users/can/write/to
> export CATALOG_QT_DB_PROPERTIES=${CATALOG_QT_HOME}/dbAPI
 
# For tcsh
> setenv CATALOG_QT_HOME `pwd`
> setenv CATALOG_QT_STORAGE /this/is/the/location/where/all/users/can/write/to
> setenv CATALOG_QT_DB_PROPERTIES ${CATALOG_QT_HOME}/dbAPI
 
# setting target machine
#
# unfortunately, there are few places where machine dependent locations are used
# at the moment, two options are available:
#
# setenv MACHINE gateway
# setenv MACHINE jet
#
# it might be, we need yet another one, with new machine settings
 
# make sure to create and properly configure gSOAP module
# this way, you will be able to use pkg-config, etc.
> module load gsoap/2.8
> make


Installation of UI - web client

Make sure you can access repository: https://gforge6.eufus.eu/svn/catalog_qt

Code Block
> svn co https://gforge6.eufus.eu/svn/catalog_qt

The tool requires the following components to work:

  • MySQL database management system
  • Apache HTTP Server
  • PHP server-side scripting language

Installation of server tools

The administrator should install the following packages on the server:

  • mysql-server or mariadb-server
  • php
  • libapache2-mod-php
  • apache2

Installation of web client

UI codes are available as a sub-project in the src directory

Code Block
src
|-- api_1_0			- API routines
|-- client			- nusoap wrappers
|-- config.php		- main configuration file
|-- inc				- auxiliary scripts
|-- nusoap			- nusoap library
|-- test.php		- nusoap testing script
`-- ui2				- main UI scripts and web page

To install the UI, copy the src folder to the apache directory (usually it is  /var/www/html/, but it depends on the version and type of operating system)
It is best to add an alias in the httpd.conf file to access the page after a short link without entering the entire address.

After all, restart the apache server:

Code Block
> service apache2 restart

Configuration of UI

the fastest way:

Because the codes are mainly prepared for operation on the GW machine, to complete the configuration, all the names of the GW server (http://portal.eufus.euhttps://portal.eufus.eu) should be replaced with the names of the new server.
These names should be located and be replaced in these files:

Code Block
./src/ui2/config.php
./src/config.php

You can configure access to the mysql database in a ./src/ui2/config.php file (The data should match those given in the section Creating db schema and Installation of MySQL/MariaDB).

Also configuration of API databases access should be similarly adapted:

Code Block
./src/config_scheduler_db.php

After all, restart the apache server:

Code Block
> service apache2 restart

Access to databases by API

The latest version of the API allows communication with many databases through one unified interface.
Thanks to this, there is no need to create a separate API instance for each database on the server side.
The client can query one WS address to have access to all databases available on the server.
In the client's SOAP request header, the variable Qdb contains the database name.
The diagram below shows how this mechanism works:

Mermaid
sequenceDiagram
    Client->>Catalog API: SOAP request [header with Qdb variable]
	Catalog API->>Scheduler DB: ask for DB configuration
	Scheduler DB->>Catalog API: send DB configuration
	Catalog API->>Catalog API: switch to chosen DB
	Catalog API->>Chosen DB: request/command to execute
	Chosen DB->>Chosen DB: execute request/command
	Chosen DB-->>Client: send feedback

The API reads the Qdb variable from the header only if the server needs this information. Otherwise, server ignores client's header:

Mermaid
sequenceDiagram
    Client->>Catalog API: SOAP request [header with Qdb variable]
	Catalog API->>Scheduler DB: request/command to execute
	Scheduler DB->>Scheduler DB: ignore header [Qdb variable]
	Scheduler DB->>Scheduler DB: execute request/command
	Scheduler DB-->>Client: send feedback
	participant Other Databases