Asynchronous product retrieval

Note that users are required to log in for data downloading, even to download public data.

In an asynchronous download request the system creates a tar file automatically. After the request, a notification of asynchronous download is shown.

Users receive a notification (via email) once the file is prepared.

The file will be kept in disk, available for downloading up to 7 days.

Every user has a disk quota of 60GB, if the maximum disk quota is reached, the download asynchronous requests will fail. Users may contact helpdesk for assistance.

The asynchronous product retrieval mode is the one recommended for big downloads, since it allows resuming it in case the transfer fails, as indicated in Download Resume section.

Steps for downloading data using an asynchronous download:

1. Login (use your user credentials)

https://hreda.esac.esa.int/hreda-sl-tap/login

2. Retrieval request (note the parameter retrieval_access=DEFERRED )

This example is downloading all the DCMIX3 files between [2016-09-18, 2016-09-22 12:00:00].


https://hreda.esac.esa.int/hreda-sl-tap/data?retrieval_type=PRODUCT&QUERY=SELECT+file_path,+file_name+FROM+hreda.v_file_metadata+WHERE+investigationt_name='DCMIX-3'+AND+acquisition_time>='2016-09-18'+AND+acquisition_time<'2016-09-18 12:00:00'+AND+active='true'&retrieval_access=DEFERRED

NOTE: It redirects to the URL of the status of the job running at server side. The status is showed at the <uws:phase> element.


Asynchronous downloads are generated by adding the parameter retrieval_access=DEFERRED to any synchronous tap download request as those defined in the generic examples section.


3a. Wait to receive the email notification.

User receives an email with subject "Your HREDA Science Archive Data request <request ID> has been processed."


4a. Get the results.

Use the link at the email body to start the download (remember that this one can be resumed if transfer fails)


Optionally

User can check the status and download the result with direct HTTP access:


3b. On the page of job status (redirected from step 2) check that the request has finished (i.e. <uws:phase> is COMPLETED). If required, reload the page until <uws:phase> is not 'EXECUTING' anymore.

https://hreda.esac.esa.int/hreda-sl-tap/tap/async/[requestID]

4b. Get the results.

Use the xlink:href got at the previous step to start the download (remember that this one can be resumed if transfer fails). This element can be found at the bottom of the page, whithin the <uws:results> element.

https://hreda.esac.esa.int/hreda-sl-tap/tap/async/[requestID]/results/[userNameRequestID]

HTTP Parameters

Parameter name Description Mandatory Possible Values
retrieval_type Type of product retrieval (by now only PRODUCT can be used) YES PRODUCT
query ADQL query allowing to download the desired file, composed of the following:
  • The columns in the SELECT clause shall always contain the file_path and file_name
  • The target in the FROM clause shall be any combination of tables with products, that is: hreda.v_file_metadata
YES See examples on the next section: Generic examples
retrieval_access The type of access retrieval. DEFERRED indicates an asynchronous download. By default, DIRECT NO DIRECT or DEFERRED
compress If true the requested product(s) will be compressed. By default, FALSE NO TRUE or FALSE
uncompress If true a single compressed file will be uncompressed. By default, FALSE NO TRUE or FALSE