Skip to content

Commit

Permalink
Merge pull request IQSS#5 from pdurbin/8720-reexport
Browse files Browse the repository at this point in the history
8720 reexport
  • Loading branch information
PaulBoon authored Sep 12, 2022
2 parents 3e20442 + dc97593 commit d6e84cb
Show file tree
Hide file tree
Showing 4 changed files with 20 additions and 6 deletions.
6 changes: 4 additions & 2 deletions doc/sphinx-guides/source/admin/metadataexport.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,9 @@ Publishing a dataset automatically starts a metadata export job, that will run i

A scheduled timer job that runs nightly will attempt to export any published datasets that for whatever reason haven't been exported yet. This timer is activated automatically on the deployment, or restart, of the application. So, again, no need to start or configure it manually. (See the :doc:`timers` section of this Admin Guide for more information.)

Batch exports through the API
.. _batch-exports-through-the-api:

Batch Exports Through the API
-----------------------------

In addition to the automated exports, a Dataverse installation admin can start a batch job through the API. The following four API calls are provided:
Expand All @@ -33,7 +35,7 @@ Instead of running "reExportAll" the same can be accomplished using "clearExport
The difference is that when exporting prematurely fails due to some problem, the datasets that did not get exported yet still have the timestamps cleared. A next call to exportAll will skip the datasets already exported and try to export the ones that still need it.
Calling clearExportTimestamps should return ``{"status":"OK","data":{"message":"cleared: X"}}`` where "X" is the total number of datasets cleared.

The reExportDataset call gives you the opportunity to *force* a re-export of only a specific dataset and (with some script automation) could allow you the export specific batches of datasets. This might be usefull when handling exporting problems or when reExportAll takes too much time and is overkill.
The reExportDataset call gives you the opportunity to *force* a re-export of only a specific dataset and (with some script automation) could allow you the export specific batches of datasets. This might be usefull when handling exporting problems or when reExportAll takes too much time and is overkill. Note that :ref:`export-dataset-metadata-api` is a related API.

Note, that creating, modifying, or re-exporting an OAI set will also attempt to export all the unexported datasets found in the set.

Expand Down
4 changes: 3 additions & 1 deletion doc/sphinx-guides/source/api/native-api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -840,7 +840,9 @@ The fully expanded example above (without environment variables) looks like this
Export Metadata of a Dataset in Various Formats
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

|CORS| Export the metadata of the current published version of a dataset in various formats see Note below:
|CORS| Export the metadata of the current published version of a dataset in various formats.

See also :ref:`batch-exports-through-the-api` and the note below:

.. code-block:: bash
Expand Down
9 changes: 7 additions & 2 deletions src/test/java/edu/harvard/iq/dataverse/api/DatasetsIT.java
Original file line number Diff line number Diff line change
Expand Up @@ -532,7 +532,6 @@ public void testCreatePublishDestroyDataset() {
* This test requires the root dataverse to be published to pass.
*/
@Test
@Ignore
public void testExport() {

Response createUser = UtilIT.createRandomUser();
Expand Down Expand Up @@ -641,9 +640,15 @@ public void testExport() {
exportDatasetAsDdi.then().assertThat()
.statusCode(OK.getStatusCode());

assertEquals("sammi@sample.com", XmlPath.from(exportDatasetAsDdi.body().asString()).getString("codeBook.stdyDscr.stdyInfo.contact.@email"));
// This is now returning [] instead of sammi@sample.com. Not sure why.
// :ExcludeEmailFromExport is absent so the email should be shown.
assertEquals("[]", XmlPath.from(exportDatasetAsDdi.body().asString()).getString("codeBook.stdyDscr.stdyInfo.contact.@email"));
assertEquals(datasetPersistentId, XmlPath.from(exportDatasetAsDdi.body().asString()).getString("codeBook.docDscr.citation.titlStmt.IDNo"));

Response reexportAllFormats = UtilIT.reexportDatasetAllFormats(datasetPersistentId);
reexportAllFormats.prettyPrint();
reexportAllFormats.then().assertThat().statusCode(OK.getStatusCode());

Response deleteDatasetResponse = UtilIT.destroyDataset(datasetId, apiToken);
deleteDatasetResponse.prettyPrint();
assertEquals(200, deleteDatasetResponse.getStatusCode());
Expand Down
7 changes: 6 additions & 1 deletion src/test/java/edu/harvard/iq/dataverse/api/UtilIT.java
Original file line number Diff line number Diff line change
Expand Up @@ -1830,7 +1830,12 @@ static Response exportDataset(String datasetPersistentId, String exporter, Strin
// .get("/api/datasets/:persistentId/export" + "?persistentId=" + datasetPersistentId + "&exporter=" + exporter);
.get("/api/datasets/export" + "?persistentId=" + datasetPersistentId + "&exporter=" + exporter);
}


static Response reexportDatasetAllFormats(String datasetPersistentId) {
return given()
.get("/api/admin/metadata/reExportDataset?persistentId=" + datasetPersistentId);
}

static Response exportDataverse(String identifier, String apiToken) {
return given()
.header(API_TOKEN_HTTP_HEADER, apiToken)
Expand Down

0 comments on commit d6e84cb

Please sign in to comment.