• Created by Unknown User (tarekkh), last modified on 08 Jul 2016

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 23 Next »

DPS description :

 

Deployed Toplogies:

Two topologies are deployed :

  • XSLT topology: used to transform an XML file based on XSL tranformation file.
  • Image conversion topology: used to convert TIFF type images into JP2 type images.

DPS usage tutorial :

This tutorial covers the usage of DPS-Rest APIs:

1. An admin should grant the user of the topology the ability to interact with a specific topology by granting this user read and write permissions on the requested topology. In the following command we are granting user: topology_user_test the permission to interact with ic topology.

 

curl -i -X POST --user admin:admin -H "Content-Type: application/x-www-form-urlencoded" -d 'username=topology_user_test' http://cloud.europeana.eu/services/image_convert/permit

 

2. Submit a task to dps;

Two types of DPS tasks could be sent :

    • A task to convert/transform list of files .
    • A task to convert/transform files inside a list of datasets.

Make sure that the required fields are included in your task or a method not allowed exception will be thrown and the task will not be sent!.

A.    For IC topology:

    • Submit a list of files:

       The required fields are:

       FILE_URLS as a data entry. And the following parameters: 
      • MIME_TYPE : The mime type of the image which will be converted.
      • OUTPUT_MIME_TYPE : The expected mime type of the output image.

      • TASK_SUBMITTER_NAME : The owner of the converted image.

        curl -X POST --user topology_user_test:topology_user_test -H "Content-Type: application/json" -d '{"inputData":{"entry":[{"key":"FILE_URLS","value":"http://cloud.europeana.eu/api/records/CLOUD_ID/representations/REPRSENTATION_NAME/versions/VERSION/files/FILENAME1 http://cloud.europeana.eu/api/records/CLOUD_ID/representations/REPRSENTATION_NAME/versions/VERSION/files/FILENAME2 http://cloud.europeana.eu/api/records/CLOUD_ID/representations/REPRSENTATION_NAME/versions/VERSION/files/FILENAME3 "}]},"parameters":{"entry":[{"key":"TASK_SUBMITTER_NAME","value":"topology_user_test"},{"key":"OUTPUT_MIME_TYPE","value":"image/jp2"},{"key":"MIME_TYPE","value":"image/tiff"}]}}' -i http://cloud.europeana.eu/services/image_convert/tasks

    • Submit a list of datasets:

       DATASET_URLS as a data entry. And the following parameters: 

      • MIME_TYPE : The mime type of the image which will be converted.
      • OUTPUT_MIME_TYPE : The expected mime type of the output image.

      • TASK_SUBMITTER_NAME : The owner of the converted image.

        curl -X POST --user topology_user_test:topology_user_test -H "Content-Type: application/json" -d '{"inputData": {"entry": [{"key": "DATASET_URLS","value": "http://cloud.europeana.eu/api/data-providers/DATA_PROVIDER1/data-sets/dDATASET_NAME1 http://cloud.europeana.eu/api/data-providers/DATA_PROVIDER2/data-sets/dDATASET_NAME2 http://cloud.europeana.eu/api/data-providers/DATA_PROVIDER3/data-sets/dDATASET_NAME3"}]},"parameters": {"entry": [{"key": "TASK_SUBMITTER_NAME","value": "admin"}, {"key": "OUTPUT_MIME_TYPE", "value": "image/jp2"}, {"key": "MIME_TYPE","value": "image/tiff"}]}}' -i http://cloud.europeana.eu/services/image_convert/tasks



B.    For XSLT topology:

      • Submit a list of files:

         The required fields are:

         FILE_URLS as a data entry. And the following parameters:
      • XSLT_URL: a URL for XSLT file which will be used to transform  XML documents.

      • TASK_SUBMITTER_NAME: The owner of the transformed files.

        curl -X POST --user topology_user_test:topology_user_test -H "Content-Type: application/json" -d '{"inputData":{"entry":[{"key":"FILE_URLS","value":"http://cloud.europeana.eu/api/records/CLOUD_ID/representations/REPRSENTATION_NAME/versions/VERSION/files/FILENAME1 http://cloud.europeana.eu/api/records/CLOUD_ID/representations/REPRSENTATION_NAME/versions/VERSION/files/FILENAME2 http://cloud.europeana.eu/api/records/CLOUD_ID/representations/REPRSENTATION_NAME/versions/VERSION/files/FILENAME3"}]},"parameters":{"entry":[{"key":"TASK_SUBMITTER_NAME","value":"topology_user_test"},{"key":"XSLT_URL","value":"http://cloud.europeana.eu/api/records/CLOUD_ID/representations/REPRSENTATION_NAME/versions/VERSION/files/XSLT_FILE_NAME"}]}}' -i http://cloud.europeana.eu/services/xslt_transform/tasks

    • Submit a list of datasets:

       DATASET_URLS as a data entry. And the following parameters: 

      • XSLT_URL: a URL for XSLT file which will be used to transform  XML documents.

      • TASK_SUBMITTER_NAME: The owner of the transformed files.

        curl -X POST --user topology_user_test:topology_user_test -H "Content-Type: application/json" -d '{"inputData":{"entry":[{"key":"DATASET_URLS","value":"http://cloud.europeana.eu/api/data-providers/DATA_PROVIDER1/data-sets/dDATASET_NAME1 http://cloud.europeana.eu/api/data-providers/DATA_PROVIDER2/data-sets/dDATASET_NAME2 http://cloud.europeana.eu/api/data-providers/DATA_PROVIDER3/data-sets/dDATASET_NAME3"}]},"parameters":{"entry":[{"key":"TASK_SUBMITTER_NAME","value":"topology_user_test"},{"key":"XSLT_URL","value":"http://cloud.europeana.eu/api/records/CLOUD_ID/representations/REPRSENTATION_NAME/versions/VERSION/files/XSLT_FILE_NAME"}]}}' -i http://cloud.europeana.eu/services/xslt_transform/tasks

      If a task was submitted successfully the sender should recieve a response confirming the creation of the task, like this:           

 

201 Created
Server: Apache-Coyote/1.1
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: 0
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-XSS-Protection: 1; mode=block
Location: http://cloud.europeana.eu/services/image_convert/tasks/TASK_ID
Content-Length: 0
Date: creation Date

     The location filed should have a url to the newely created task and at the end of this url a TASK_ID should exist. This taskID should be used to follow the progress of this task.


3. Reporting: 

To monitor the progress of your tasks.

Reporting is divided into two requests:  

    • Report on a task level:

       curl --user topology_user_test:topology_user_test http://cloud.europeana.eu/services/image_convert/tasks/TASK_ID/progress

      This call should give you information about the status of the submittted task like the status of the task (SENT-PROCESSED-CURRENTLY_PROCESSED,PENDING,DROPPED), total number of files inside a task, the number of the processed files , the exact time stating when task was sent, start processing and finished .

      {"taskId":TASK_ID,"topologyName":"ic_topology","totalSize":2,"processed":2,"state":"PROCESSED","info":"Completely processed","sent_time":"2016-27-07 09:27:40","start_time":"2016-28-07 09:28:12","finish_time":"2016-28-07 09:28:12"}

           

    • detailed information about every file in a task. 

       curl --user topology_user_test:topology_user_test http://cloud.europeana.eu/services/image_convert/tasks/TASK_ID/notification

      [{"task_id":TASK_ID,"topology_name":"ic_topology","resource":"http://cloud.europeana.eu/api/records/CLOUD_ID/representations/REPRSENTATION_NAME/versions/VERSION/files/FILENAME1","state":"SUCCESS","info_text":"","additional_informations":"","result_resource":"

      http://cloud.europeana.eu/api/records/CLOUD_ID/representations/NEW_REPRSENTATION_NAME/versions/NEW_VERSION/files/NEW_FILENAME1"},{"task_id":TASK_ID,"topology_name":"ic_topology","resource":" http://cloud.europeana.eu/api/records/CLOUD_ID/representations/REPRSENTATION_NAME/versions/VERSION/files/FILENAME2","state":"SUCCESS","info_text":"","additional_informations":"","result_resource":"
      http://cloud.europeana.eu/api/records/CLOUD_ID/representations/NEW_REPRSENTATION_NAME/versions/NEW_VERSION/files/NEW_FILENAME1"}]

 


 

  • No labels