You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 67 Current »

This tutorial covers the usage of DPS REST API

 

An admin should grant the user of the topology the ability to interact with a specific topology by granting this user read and write permissions on the requested topology.

Submit a task to DPS


Two types of DPS tasks could be sent :

  • A task to convert/transform list of files.
  • A task to convert/transform files inside a list of datasets.

Make sure that the required fields are included in your task or a method not allowed exception will be thrown and the task will not be sent!.

 For Image Conversion topology 

  • Submit a list of files:

     The required fields are:

     FILE_URLS as a data entry. And the following parameters: 
    • MIME_TYPE : The mime type of the image which will be converted.
    • OUTPUT_MIME_TYPE : The expected mime type of the output image.

    • TASK_SUBMITTER_NAME : The owner of the converted image.

      curl -X POST --user topology_user_test:topology_user_test -H "Content-Type: application/json" -d '{"inputData":{"entry":[{"key":"FILE_URLS","value":"http://cloud.europeana.eu/api/records/CLOUD_ID/representations/REPRSENTATION_NAME/versions/VERSION/files/FILENAME1 http://cloud.europeana.eu/api/records/CLOUD_ID/representations/REPRSENTATION_NAME/versions/VERSION/files/FILENAME2 http://cloud.europeana.eu/api/records/CLOUD_ID/representations/REPRSENTATION_NAME/versions/VERSION/files/FILENAME3 "}]},"parameters":{"entry":[{"key":"TASK_SUBMITTER_NAME","value":"topology_user_test"},{"key":"OUTPUT_MIME_TYPE","value":"image/jp2"},{"key":"MIME_TYPE","value":"image/tiff"}]}}' -i http://cloud.europeana.eu/services/image_convert/tasks

  • Submit a list of datasets:

     DATASET_URLS as a data entry. And the following parameters: 

    • MIME_TYPE : The mime type of the image which will be converted.
    • OUTPUT_MIME_TYPE : The expected mime type of the output image.

    • TASK_SUBMITTER_NAME : The owner of the converted image.

      curl -X POST --user topology_user_test:topology_user_test -H "Content-Type: application/json" -d '{"inputData": {"entry": [{"key": "DATASET_URLS","value": "http://cloud.europeana.eu/api/data-providers/DATA_PROVIDER1/data-sets/dDATASET_NAME1 http://cloud.europeana.eu/api/data-providers/DATA_PROVIDER2/data-sets/dDATASET_NAME2 http://cloud.europeana.eu/api/data-providers/DATA_PROVIDER3/data-sets/dDATASET_NAME3"}]},"parameters": {"entry": [{"key": "TASK_SUBMITTER_NAME","value": "admin"}, {"key": "OUTPUT_MIME_TYPE", "value": "image/jp2"}, {"key": "MIME_TYPE","value": "image/tiff"}]}}' -i http://cloud.europeana.eu/services/image_convert/tasks



For XSLT topology:

  • Submit a list of files:

     The required fields are:

     FILE_URLS as a data entry. And the following parameters:
    • XSLT_URL: a URL for XSL file which will be used to transform  XML documents.

    • TASK_SUBMITTER_NAME: The owner of the transformed files. 

      curl -X POST --user topology_user_test:topology_user_test -H "Content-Type: application/json" -d '{"inputData":{"entry":[{"key":"FILE_URLS","value":"http://cloud.europeana.eu/api/records/CLOUD_ID/representations/REPRSENTATION_NAME/versions/VERSION/files/FILENAME1 http://cloud.europeana.eu/api/records/CLOUD_ID/representations/REPRSENTATION_NAME/versions/VERSION/files/FILENAME2 http://cloud.europeana.eu/api/records/CLOUD_ID/representations/REPRSENTATION_NAME/versions/VERSION/files/FILENAME3"}]},"parameters":{"entry":[{"key":"TASK_SUBMITTER_NAME","value":"topology_user_test"},{"key":"XSLT_URL","value":"http://cloud.europeana.eu/api/records/CLOUD_ID/representations/REPRSENTATION_NAME/versions/VERSION/files/XSLT_FILE_NAME"}]}}' -i http://cloud.europeana.eu/services/xslt_transform/tasks

  • Submit a list of datasets:

     DATASET_URLS as a data entry. And the following parameters: 

    • XSLT_URL: a URL for XSL file which will be used to transform  XML documents.

    • TASK_SUBMITTER_NAME: The owner of the transformed files.  

      curl -X POST --user topology_user_test:topology_user_test -H "Content-Type: application/json" -d '{"inputData":{"entry":[{"key":"DATASET_URLS","value":"http://cloud.europeana.eu/api/data-providers/DATA_PROVIDER1/data-sets/dDATASET_NAME1 http://cloud.europeana.eu/api/data-providers/DATA_PROVIDER2/data-sets/dDATASET_NAME2 http://cloud.europeana.eu/api/data-providers/DATA_PROVIDER3/data-sets/dDATASET_NAME3"}]},"parameters":{"entry":[{"key":"TASK_SUBMITTER_NAME","value":"topology_user_test"},{"key":"XSLT_URL","value":"http://cloud.europeana.eu/api/records/CLOUD_ID/representations/REPRSENTATION_NAME/versions/VERSION/files/XSLT_FILE_NAME"}]}}' -i http://cloud.europeana.eu/services/xslt_transform/tasks

Notes:

  •  If a task was submitted successfully the sender should recieve a response confirming the creation of the task, like this:  

     

    201 Created
Server: Apache-Coyote/1.1
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: 0
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-XSS-Protection: 1; mode=block
Location: http://cloud.europeana.eu/services/image_convert/tasks/TASK_ID
Content-Length: 0
Date: creation Date

         The location filed should have a url to the newely created task and at the end of this url a TASK_ID should exist. This taskID should be used to follow the progress of this task.

     

  •       There is a list of optional parameters which could be used in a DPS task :          
    • REPRESENTATION_NAME: If a task designed to process specific representation inside a dataset.

    • NEW_REPRESENTATION_NAME: The name of the representation used to store the processed files.        


 Reporting 

You can monitor the progress of your tasks.

 Reporting is divided into two requests:  

    • Report on a task level:

       curl --user topology_user_test:topology_user_test http://cloud.europeana.eu/services/image_convert/tasks/TASK_ID/progress

      This call should give you information about the status of the submittted task like the status of the task (SENT-PROCESSED-CURRENTLY_PROCESSED,PENDING,DROPPED), total number of files inside a task, the number of the processed files , the exact time stating when task was sent, started processing and finished . An example response :

      {"taskId":TASK_ID,"topologyName":"ic_topology","totalSize":2,"processed":2,"state":"PROCESSED","info":"Completely processed","sent_time":"2016-27-07 09:27:40","start_time":"2016-28-07 09:28:12","finish_time":"2016-28-07 09:28:12"}

    • Detailed information about every file in a task. 

       curl --user topology_user_test:topology_user_test http://cloud.europeana.eu/services/image_convert/tasks/TASK_ID/notification

      An example response :

      [{"task_id":TASK_ID,"topology_name":"ic_topology","resource":"http://cloud.europeana.eu/api/records/CLOUD_ID/representations/REPRSENTATION_NAME/versions/VERSION/files/FILENAME1",
"state":"SUCCESS","info_text":"","additional_informations":"","result_resource":"http://cloud.europeana.eu/api/records/CLOUD_ID/representations/NEW_REPRSENTATION_NAME/versions/NEW_VERSION/files/NEW_FILENAME1"},{"task_id":TASK_ID,"topology_name":"ic_topology","resource":" http://cloud.europeana.eu/api/records/CLOUD_ID/representations/REPRSENTATION_NAME/versions/VERSION/files/FILENAME2",
"state":"SUCCESS","info_text":"","additional_informations":"","result_resource":"http://cloud.europeana.eu/api/records/CLOUD_ID/representations/NEW_REPRSENTATION_NAME/versions/NEW_VERSION/files/NEW_FILENAME2"}]

 


 

  • No labels