| Note | |
|---|---|
|
| Code Block | ||
|---|---|---|
| ||
|
...
| ||
|
...
|
...
|
...
|
Data Providers
Each piece of data uploaded to Europeana Cloud has to be owned by a data provider. You can manage the data providers - create or update them - using the API. Let's start with creating one. This is done using the /data-providers method.
Creation
In order to create a data provider you need to use this request: https://cloud.europeana.eu/docs/now/uis/index.html#295750320. . You need to provide a unique identifier for the provider in the providerId parameter. You can specify additional properties and send them as JSON in data block:
...
| Code Block | ||
|---|---|---|
| ||
curl -X POST -H "Content-Type: application/json" --user user:password -d '{
"organisationWebsiteURL": "http://www.man.poznan.pl",
"organisationWebsite": "Poznańskie Centrum Superkomputerowo - Sieciowe Afiliowane przy Instytucie Chemii Bioorganicznej PANPoznan Supercomputing and Networking Center Affiliated to Institute of Bioorganic Chemistry Polish Academy of Science",
"officialAddress" : "ul. Jana Pawła II 10, Poznań",
"digitalLibraryURL" : "http://lib.psnc.pl/dlibra",
"digitalLibraryWebsite": "RepozytoriumInstitutional InstytucjonalneRepository",
"organisationName": "PoznańskiePoznan Centrum Superkomputerowo - Sieciowe Afiliowane przy Instytucie Chemii Bioorganicznej PANSupercomputing and Networking Center Affiliated to Institute of Bioorganic Chemistry Polish Academy of Science",
"remarks": "Integracja i rozwój infrastruktury informatycznej naukiIntegrating and developing the information infrastructure for science",
"contactPerson": "office@man.poznan.pl"
}' -i https://cloudecloud.europeanapsnc.eupl/api/data-providers?providerId=PSNC |
...
If the provider was created successfully you will get a response with HTTP code 200. You can use the GET method (https://cloud.europeana.eu/docs/now/uis/index.html#-927343388) on the same endpoint to make sure on the same endpoint to make sure that the system returns information about the data provider, when queried by its id:
| Code Block | ||
|---|---|---|
| ||
curl -X GET -H "Accept: application/json" --user user:password -i https://cloudecloud.europeanapsnc.eupl/api/data-providers/PSNC |
The response should look like this:
| Code Block | ||
|---|---|---|
| ||
{
"active":true,
"id":"PSNC",
"partitionKey":2465528,
"properties":{
"contactPerson":"office@man.poznan.pl",
"digitalLibraryURL":"http://lib.psnc.pl/dlibra",
"digitalLibraryWebsite":"RepozytoriumInstitutional InstytucjonalneRepository",
"officialAddress":"ul. Jana Pawła II 10, Poznań",
"organisationName":"PoznańskiePoznan CentrumSupercomputing Superkomputerowoand -Networking SiecioweCenter AfiliowaneAffiliated przyto InstytucieInstitute Chemiiof Bioorganicznej PAN",
Bioorganic Chemistry Polish Academy of Science",
"organisationWebsite":"Poznańskie Centrum Superkomputerowo - Sieciowe Afiliowane przy Instytucie Chemii Bioorganicznej PANPoznan Supercomputing and Networking Center Affiliated to Institute of Bioorganic Chemistry Polish Academy of Science",
"organisationWebsiteURL":"http://www.man.poznan.pl",
"remarks":"Integracja i rozwój infrastruktury informatycznej naukiIntegrating and developing the information infrastructure for science"
}
} |
Identifiers
Creation
Every record has a unique identifier in Europeana Cloud. We call these identifiers "global". A record can also have an identifier in its original the system; these ïdentifiers identifiers are referred to as “local”.
Prior to uploading data to a record, the system needs to generate its global identifier and map it to the local one. You can do this by making this request.
| Code Blockcode | ||
|---|---|---|
| ||
curl -X POST --user user:password -i https://cloudecloud.europeanapsnc.eupl/api/cloudIds?providerId=exampleProviderPSNC\&recordId=exampleLocalRecord |
The response will contain the mapping between the local record and the newly created identifier LE27IU4ZYQP4OQU2EJLCT2LSSDZO2YVIJYX27JWT4RN7PQNF76IA:
...
| Note |
|---|
Have you noticed the "\&" before recordId parameter? the & character had to be escaped because it is a special character in bash. You might not need it if you use another interpreter or HTTP client. |
The response will contain the mapping between the local record and the newly created identifier X67WPA7NGCZUKLACPWU3BORRGQUWTKSZNZDICEGZS6IXTRMINOIQ:
| Code Block |
|---|
<?xml version="1.0" encoding="UTF-8"?> <cloudId> <id>X67WPA7NGCZUKLACPWU3BORRGQUWTKSZNZDICEGZS6IXTRMINOIQ</id> <localId> <providerId>PSNC</providerId> |
...
<recordId>exampleLocalRecord</recordId> |
...
</localId>
</cloudId>
|
| Code Block |
|---|
curl -X POST --user user:password https:// |
ecloud. |
psnc. |
pl/api/cloudIds?providerId=exampleProvider |
The newly created global identifier and a local one can be found in the response:
...
| Code Block |
|---|
<?xml version="1.0" encoding="UTF-8"?> <cloudId> <id>745XTNGBTQBVVFFZ7XF5QETU7ECIUGHKUXBJ3DNRIN3WCKRPOVVQ</id> <localId> |
...
<providerId>PSNC</providerId> |
...
<recordId>4TIRTYXNTQ2LNDWWVDKMK6TJJMJJM3I6WO4IBIDB6ORYXQB5TZIQ</recordId> |
...
</localId>
</cloudId> |
Listing
You can browse all the global ids associated with a specific data provider using the /cloudIds REST endpoint:
| Code Block |
|---|
curl -X GET -H "Accept: application/json" --user user:password -i https:// |
...
ecloud. |
...
psnc. |
...
pl/api/data-providers/ |
...
PSNC/cloudIds |
In the response we should get a list of all cloud identifiers assigned to our provider.
| Code Block |
|---|
{ |
...
"results":[ |
...
...
{ |
...
"type": |
...
"cloudId", |
...
"id": |
...
" |
...
745XTNGBTQBVVFFZ7XF5QETU7ECIUGHKUXBJ3DNRIN3WCKRPOVVQ", |
...
"localId":{ |
...
|
...
"providerId": |
...
" |
...
PSNC", |
...
"recordId": |
...
" |
...
4TIRTYXNTQ2LNDWWVDKMK6TJJMJJM3I6WO4IBIDB6ORYXQB5TZIQ" |
...
} |
...
}, |
...
{ |
...
"type": |
...
"cloudId", |
...
"id": |
...
" |
...
X67WPA7NGCZUKLACPWU3BORRGQUWTKSZNZDICEGZS6IXTRMINOIQ", |
...
"localId":{ |
...
|
...
"providerId": |
...
" |
...
PSNC", |
...
"recordId": |
...
"exampleLocalRecord" |
...
} |
...
} |
...
] |
...
} |
Now you are ready to fill Europeana Cloud with some data.
Records, Representations and Files
A record is the central entity of the storage system. It has a global identifier and and consists of several representations.
Representations
Lets get back to our example in the Data Model section. Say that you want to create a record out of the three files you have. Each one will be stored in a different representation. Creating new representations is done using the /records REST endpoint:
| Code Block |
|---|
curl -X POST -H "Accept: application/json" --user user:password -H "Content-Type: application/x-www-form-urlencoded" -d 'providerId= |
...
PSNC' -i https:// |
...
ecloud. |
...
psnc. |
...
pl/api/records/ |
...
X67WPA7NGCZUKLACPWU3BORRGQUWTKSZNZDICEGZS6IXTRMINOIQ/representations/TIFF |
...
curl -X POST -H "Accept: application/json" --user user:password -H "Content-Type: application/x-www-form-urlencoded" -d 'providerId= |
...
PSNC' -i https:// |
...
ecloud. |
...
psnc. |
...
pl/api/records/ |
...
X67WPA7NGCZUKLACPWU3BORRGQUWTKSZNZDICEGZS6IXTRMINOIQ/representations/thumbnail |
...
curl -X POST -H "Accept: application/json" --user user:password -H "Content-Type: application/x-www-form-urlencoded" -d 'providerId= |
...
PSNC' -i https:// |
...
ecloud. |
...
psnc. |
...
pl/api/records/ |
...
X67WPA7NGCZUKLACPWU3BORRGQUWTKSZNZDICEGZS6IXTRMINOIQ/representations/EDM |
When successful, the response will have HTTP status code 201 (Created). Note that when a new representation is created, it is automatically assigned a new version. This new version can be reached directly using a URL which is returned in the location field of the response header. The URL will be of the following format:
| Code Block |
|---|
https:// |
ecloud. |
psnc. |
pl/api/records/ |
X67WPA7NGCZUKLACPWU3BORRGQUWTKSZNZDICEGZS6IXTRMINOIQ/representations/thumbnail/versions/ |
cd9f0050- |
3dea- |
11e6- |
b03d- |
fa163e8d4ae3 |
Subsequent versions of a representation can be created further in the future, for example to reflect ongoing work on a file.
Files and Versions
You just created metadata entries for your data representations. Now it’s time to attach real data to them. For this the /versions REST endpoint is used. For example, to add a thumbnail file to a previously created representation thumbnail:
| Code Block |
|---|
curl -X POST --user user:password -H "Content-Type: multipart/form-data" -F "mimeType=application/png" -F "data=@thumbnail.png" https:// |
...
ecloud. |
...
psnc. |
...
pl/api/records/ |
...
X67WPA7NGCZUKLACPWU3BORRGQUWTKSZNZDICEGZS6IXTRMINOIQ/representations/thumbnail/versions/ |
...
cd9f0050- |
...
3dea- |
...
11e6- |
...
b03d- |
...
fa163e8d4ae3/files |
URI of the newly added file will be returned in the location field of the response header:
| Code Block |
|---|
https:// |
...
ecloud. |
...
psnc. |
...
pl/api/records/ |
...
X67WPA7NGCZUKLACPWU3BORRGQUWTKSZNZDICEGZS6IXTRMINOIQ/representations/thumbnail/versions/ |
...
cd9f0050- |
...
3dea- |
...
11e6- |
...
b03d- |
...
fa163e8d4ae3/files/ |
...
1df7a491- |
...
8d8a- |
...
4180- |
...
9eb0-cd18c52a4b81 |
You can add multiple many files to a version.
The newly created version is a temporary one, which means it can be removed. To make a version persistent, you can use the /persist call:
| Code Block |
|---|
curl -X POST --user user:password -i https:// |
...
ecloud. |
...
psnc. |
...
pl/api/records/ |
...
X67WPA7NGCZUKLACPWU3BORRGQUWTKSZNZDICEGZS6IXTRMINOIQ/representations/ |
...
thumbnail/versions/ |
...
cd9f0050- |
...
3dea- |
...
11e6- |
...
b03d- |
...
fa163e8d4ae3/persist |
A successful call will return 201 HTTP Code.
Deletion
Versions can be deleted using the DELETE call as follows:
| Code Block |
|---|
curl -X DELETE --user user:password |
...
https:// |
...
ecloud. |
...
psnc. |
...
pl/api/records/ |
...
X67WPA7NGCZUKLACPWU3BORRGQUWTKSZNZDICEGZS6IXTRMINOIQ/representations/thumbnail/versions/ |
...
cd9f0050- |
...
3dea- |
...
11e6- |
...
b03d- |
...
fa163e8d4ae3/ |
You can only delete a temporary version of a representation. If you will try to remove a persistent version you will get HTTP status code 405 (Not Allowed).
Datasets
Datasets in Europeana Cloud are a way to group records for thematic or operational reasons. One record can belong to many datasets.
Creation
To create a new dataset, use the /dataset call where you specify the name of the dataset.
| Code Block |
|---|
curl -X POST --user user:password -H "Content-Type: application/x-www-form-urlencoded" -d 'dataSetId=exampleDataSet' -i https:// |
...
ecloud. |
...
psnc. |
...
pl/api/data-providers/ |
...
PSNC/data-sets |
Modification
Having created one, you can assign your representation versions to this dataset. To do this use the /assignments call providing the global id, the representation name and the version identifier. If you wondon't pass provide the version identifier parameter, the latest persistent version will be assigned to data set.
| Code Block |
|---|
curl -X POST --user user:password -H "Content-Type: application/x-www-form-urlencoded" -d 'cloudId= |
...
X67WPA7NGCZUKLACPWU3BORRGQUWTKSZNZDICEGZS6IXTRMINOIQ&representationName=thumbnail&version= |
...
cd9f0050- |
...
3dea- |
...
11e6- |
...
b03d- |
...
fa163e8d4ae3' -i |
...
https://ecloud.psnc.pl/api/data-providers/PSNC/data-sets/exampleDataSet/assignments |
To remove a representation version from a dataset, use the /assignments DELETE method when you specify the dataset and representation:
| Code Block |
|---|
curl -X DELETE --user user:password -H "Content-Type: application/x-www-form-urlencoded" -i https://ecloud.psnc.pl/api/data-providers/PSNC/data-sets/exampleDataSet/assignments?cloudId=X67WPA7NGCZUKLACPWU3BORRGQUWTKSZNZDICEGZS6IXTRMINOIQ\&representationName=thumbnail |
Listing
To view the content of a dataset, use the /data-sets REST endpoint when providing the data provider id and the dataset id.
| Code Block |
|---|
curl -X GET -H "Accept: application/xml" --user user:password -i https://ecloud.psnc.pl/api/data-providers/PSNC/data-sets/exampleDataSet/ |
Deletion
To delete the dataset use the /data-sets REST endpoint when providing the data provider id and the dataset id.
| Code Block |
|---|
curl -X DELETE -H "Accept: application/json" --user user:password -i https://ecloud.psnc.pl/api/data-providers/PSNC/data-sets/exampleDataSet/ |
Permissions
Permission Granularity
In Europeana Cloud access to data entities is governed by permissions assigned to these entities. The way these permissions represent our policies on access and rights will be based on the general principles listed in the Europeana Cloud Access Framework.
In the current version access is managed on the level of versions. That means that data owners can grant or revoke access to versions (and the files associated to the version) owned by them. In future versions we will consider extending permission management also to higher levels: representations, records or datasets.
Using Permissions
Taking our example, assume that you want to make your thumbnail files publicly available. For this use the /permit REST call to to change permissions for thumbnail representations of your records.
| Code Block |
|---|
curl -X POST --user user:password -i https://ecloud.psnc.pl/api/records/X67WPA7NGCZUKLACPWU3BORRGQUWTKSZNZDICEGZS6IXTRMINOIQ/representations/thumbnail/versions/cd9f0050-3dea-11e6-b03d-fa163e8d4ae3/permit |
If you want to give permissions to another user user:
| Code Block |
|---|
curl -X POST --user user:password https://ecloud.psnc.pl/api/records/X67WPA7NGCZUKLACPWU3BORRGQUWTKSZNZDICEGZS6IXTRMINOIQ/representations/thumbnail/versions/cd9f0050-3dea-11e6-b03d-fa163e8d4ae3/permissions/permission/users/anotherUser |
In the URL above you should replace 'permission' with one of the values: read, write, delete, administration, all. 'anotherUser' should be replaced with a username.
Friendly URLs
Sometimes it is hard for a client to get the global identifier of a record, generated version name of a representation and generated filename. On the other hand the client has got a local record identifier which is well known. The same may be in case of a filename. In order to access files without retrieving the hard to remember identifiers and names mechanism of friendly URLs was introduced. The idea behind it is to use in URLs well known values (identifiers, names, etc.) always in the same way. Here we will go through the tutorial steps using friendly URLs where it is possible.
Record
Normally when you know the cloud identifier of a record you can access it using URL like this:
| Code Block |
|---|
https://ecloud.psnc.pl/api/records/X67WPA7NGCZUKLACPWU3BORRGQUWTKSZNZDICEGZS6IXTRMINOIQ |
If you remember from the beginning of this tutorial, this record was created for PSNC data provider together with local identifier which was exampleLocalRecord. So now we have only two values: name of the data provider (PSNC) and identifier used by this data provider (exampleLocalRecord). To get access to this record in Europeana Cloud using just these two values we can use friendly URL like this:
| Code Block |
|---|
curl -X GET -H "Accept: application/json" --user user:password |
To remove a representation version from a dataset, use the /assignments REST method when you specify the dataset and representation URL:
-i https:// |
ecloud. |
psnc. |
pl/api/data-providers/ |
PSNC/ |
Listing
...
records/exampleLocalRecord |
Representation
Sometimes it is necessary to get access to a specific representation. Again, instead of retrieving the global identifier we can use well known values like data provider name, local record identifier and representation name. Let's display information about the TIFFrepresentation using the friendly URL:
| Code Block |
|---|
curl -X GET |
-H "Accept: application/json" --user user:password -i |
https:// |
ecloud. |
psnc. |
pl/api/data-providers/PSNC/ |
records/ |
exampleLocalRecord/ |
representations/TIFF |
File
Deletion
...
If you do not want to have files with names generated by the system there is possibility to specify the name in the insert request. We can add a file to TIFF representation and specify a name for it like this:
| Code Block |
|---|
curl -X |
POST --user user:password -H " |
Content-Type: |
multipart/ |
form-data" - |
F "mimeType=image/tiff" -F "data=@tiff.tiff" -F "fileName=abc" https:// |
ecloud. |
psnc. |
pl/api/records/ |
X67WPA7NGCZUKLACPWU3BORRGQUWTKSZNZDICEGZS6IXTRMINOIQ/ |
representations/ |
TIFF/ |
Permissions
Permission Granularity
In Europeana Cloud access to data entities is governed by permissions assigned to these entities. The way these permissions represent our policies on access and rights will be based on the general principles listed in the Europeana Cloud Access Framework.
In the current version access is managed on the level of files. That means that data owners can grant or revoke access to files owned by them. In future versions we will consider extending permission management also to higher levels: representations, records or datasets.
Using Permissions
Taking our example, assume that you want to make your thumbnail files publicly available. For this use the /permit REST call to to change permissions for thumbnail representations of your records.
curl -X POST --user user:password -i https://cloud.europeana.eu/api/records/9H9WXX872WL/representations/thumbnail/versions/6ab713f0-bcf5-11e4-919c-525400ea6731/permitIf you want to give permissions to another user user:
curl -X POST --user user:password https://cloud.europeana.eu/api/records/9H9WXX872WL/representations/thumbnail/versions/6ab713f0-bcf5-11e4-919c-525400ea6731/users/anotherUser/permit/permissionFuture Work and Contact
Europeana Cloud is now in its alpha phase. During the remainder of the project we’ll improve the platform and the API based on the feedback from the community.
...
versions/27ae2a60-3ebf-11e6-b03d-fa163e8d4ae3/files |
As you can see the specified filename is abc. Now we can use it to access this file in Europeana Cloud via friendly URL.
| Note |
|---|
Just remember that a file will not be accessible until you make its representation persistent. |
| Code Block |
|---|
curl -X GET -H "Accept: application/json" --user user:password -i https://ecloud.psnc.pl/api/data-providers/PSNC/records/exampleLocalRecord/representations/TIFF/abc |
Filename used instead of a generated one can be also much more complex. It can for example contain / characters like in the absolute path. We can take a look at the real example provided by Europeana Newspapers.
| Code Block |
|---|
https://ecloud.psnc.pl/api/data-providers/TheEuropeanLibrary/records/3000100349685/representations/presentation_images/node-2/image/NLE/???????????_??????????_?????????_=_Livländische_Gouvernements-Zeitung/1862/01/03/18620103_1-0001.jp2 |
As you can see the filename used here (node-2/image/NLE/???????????_??????????_?????????_=_Livländische_Gouvernements-Zeitung/1862/01/03/18620103_1-0001.jp2) is not only long, having / characters inside but also contains unicode characters. We can also extract other well known values from this URL: data provider - TheEuropeanLibrary, local record identifier - 3000100349685, representation name - presentation_images.