Expose export formats in native API #10739

julian-schneider · 2024-08-02T12:57:12Z

What this PR does / why we need it: The PR adds a method that exposes the full list of available exporters (display- and format names). This is especially useful for instances that have added custom formats.

Which issue(s) this PR closes: I know no pre-existing issues for this.

Suggestions on how to test this: New method found here: /api/info/exportFormats

Does this PR introduce a user interface change? If mockups are available, please link/include them here: No, just API and documentation changes.

API answer looks like this:

{
  "status": "OK",
  "data": [
    {
      "displayName": "OAI_ORE",
      "formatName": "OAI_ORE"
    },
    {
      "displayName": "DataCite",
      "formatName": "Datacite"
    },
    {
      "displayName": "Dublin Core",
      "formatName": "oai_dc"
    },
    {
      "displayName": "OpenAIRE",
      "formatName": "oai_datacite"
    },
    {
      "displayName": "Schema.org JSON-LD",
      "formatName": "schema.org"
    },
    {
      "displayName": "DDI",
      "formatName": "ddi"
    },
    {
      "displayName": "Dublin Core",
      "formatName": "dcterms"
    },
    {
      "displayName": "DDI HTML Codebook",
      "formatName": "html"
    },
    {
      "displayName": "JSON",
      "formatName": "dataverse_json"
    },
    {
      "displayName": "DDI",
      "formatName": "oai_ddi"
    }
  ]
}

Rendered docs entry looks like this:

Is there a release notes update needed for this change?:
New API method for listing the available exporters. Found at /api/info/exportFormats, produces display names and format names.

pdurbin

@julian-schneider good idea! Thanks for the pull request. I left you some comments.

pdurbin · 2024-08-02T13:39:33Z

src/main/java/edu/harvard/iq/dataverse/api/Info.java

+    @GET
+    @Path("exportFormats")
+    public Response getExportFormats() {
+        JsonArrayBuilder responseModel = Json.createArrayBuilder();


Did you consider returning an object instead? Just this week I pushed a commit to return MPCONFIG options via API and I played with returning an array but ended up liking object better. Here's the code: 771d85a

I'm suggesting something like this:

{ "status": "OK", "data": { "OAI_ORE": { "displayName": "OAI_ORE" }, "oai_dc": { "displayName": "Dublin Core" } } }

Alternatively, perhaps you could add a second endpoint for getting info about a single export format (e.g. /api/info/exportFormats/oai_dc). That way you can easily get info on that format instead of having to search through an array to find the element that matches oai_dc.

Just some thoughts! 😄 Maybe the array is actually better.

Yeah an object is probably better, especially with the increased amount of properties, so I switched this in cc99729

pdurbin · 2024-08-02T13:45:16Z

src/main/java/edu/harvard/iq/dataverse/api/Info.java

+    @Path("exportFormats")
+    public Response getExportFormats() {
+        JsonArrayBuilder responseModel = Json.createArrayBuilder();
+        ExportService.getInstance().getExportersLabels().forEach(labels -> responseModel.add(Json.createObjectBuilder().add("displayName", labels[0]).add("formatName", labels[1])));


Is displayName and formatName enough? While we're doing this, should we add isHarvestable, mediaType, and isAvailableToUsers (isVisibleInUserInterface might be better)?

For XML exporters there even these methods:

public String getXMLNameSpace(); public String getXMLSchemaLocation(); public String getXMLSchemaVersion();

In cc99729 the additional properties you mentioned have been added, thanks for the recommendation!

pdurbin · 2024-08-02T13:46:14Z

doc/sphinx-guides/source/api/native-api.rst

@@ -4618,6 +4618,25 @@ The fully expanded example above (without environment variables) looks like this



The docs look great. Can you please add a release note snippet? Please see https://guides.dataverse.org/en/6.3/developers/version-control.html#writing-a-release-note-snippet

Has been added in 3f81981.

pdurbin · 2024-08-02T13:47:10Z

doc/sphinx-guides/source/api/native-api.rst

+Get Export Formats
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Get the available export formats. The response contains a list of objects each containing the display name and format name for an available exporter.


Does this include external exporters? I assume so. Can we please mention they are included as well?

As far as I know, all exporters, standard and custom, are managed by the ExportService, and are consequently all listed. I improved the docs in 3b2c8e5.

pdurbin · 2024-08-02T13:49:19Z

src/main/java/edu/harvard/iq/dataverse/api/Info.java

+        ExportService.getInstance().getExportersLabels().forEach(labels -> responseModel.add(Json.createObjectBuilder().add("displayName", labels[0]).add("formatName", labels[1])));
+        return ok(responseModel);
+    }
+
    private Response getSettingResponseByKey(SettingsServiceBean.Key key) {
        String setting = settingsService.getValueForKey(key);
        if (setting != null) {


I'll just put this comment at the bottom. An API test would be nice. I can help you get started if you need it.

A basic API test has been added in 95d34c8, which compares the response to what should be expected from a Dataverse installation with the default set of exporters. Is this what you had in mind?

julian-schneider · 2024-08-02T14:20:13Z

Thanks for your feedback @pdurbin - I will be absent for two weeks, so some time will pass before I revise this. See you then!

…ing response as JSON object

julian-schneider · 2024-08-28T09:15:53Z

The response is now an object, and contains more properties per format, as suggested.

New response looks like this:

(expand response)

{
  "status": "OK",
  "data": {
    "OAI_ORE": {
      "displayName": "OAI_ORE",
      "mediaType": "application/json",
      "isHarvestable": false,
      "isVisibleInUserInterface": true
    },
    "Datacite": {
      "displayName": "DataCite",
      "mediaType": "application/xml",
      "isHarvestable": true,
      "isVisibleInUserInterface": true,
      "XMLNameSpace": "http://datacite.org/schema/kernel-3",
      "XMLSchemaLocation": "http://datacite.org/schema/kernel-3 http://schema.datacite.org/meta/kernel-3/metadata.xsd",
      "XMLSchemaVersion": "3.0"
    },
    "oai_dc": {
      "displayName": "Dublin Core",
      "mediaType": "application/xml",
      "isHarvestable": true,
      "isVisibleInUserInterface": false,
      "XMLNameSpace": "http://www.openarchives.org/OAI/2.0/oai_dc/",
      "XMLSchemaLocation": "http://www.openarchives.org/OAI/2.0/oai_dc.xsd",
      "XMLSchemaVersion": "2.0"
    },
    "oai_datacite": {
      "displayName": "OpenAIRE",
      "mediaType": "application/xml",
      "isHarvestable": true,
      "isVisibleInUserInterface": true,
      "XMLNameSpace": "http://datacite.org/schema/kernel-4",
      "XMLSchemaLocation": "http://schema.datacite.org/meta/kernel-4.1/metadata.xsd",
      "XMLSchemaVersion": "4.1"
    },
    "schema.org": {
      "displayName": "Schema.org JSON-LD",
      "mediaType": "application/json",
      "isHarvestable": false,
      "isVisibleInUserInterface": true
    },
    "ddi": {
      "displayName": "DDI",
      "mediaType": "application/xml",
      "isHarvestable": false,
      "isVisibleInUserInterface": true,
      "XMLNameSpace": "ddi:codebook:2_5",
      "XMLSchemaLocation": "https://ddialliance.org/Specification/DDI-Codebook/2.5/XMLSchema/codebook.xsd",
      "XMLSchemaVersion": "2.5"
    },
    "dcterms": {
      "displayName": "Dublin Core",
      "mediaType": "application/xml",
      "isHarvestable": false,
      "isVisibleInUserInterface": true,
      "XMLNameSpace": "http://purl.org/dc/terms/",
      "XMLSchemaLocation": "http://dublincore.org/schemas/xmls/qdc/dcterms.xsd",
      "XMLSchemaVersion": "2.0"
    },
    "html": {
      "displayName": "DDI HTML Codebook",
      "mediaType": "text/html",
      "isHarvestable": false,
      "isVisibleInUserInterface": true
    },
    "dataverse_json": {
      "displayName": "JSON",
      "mediaType": "application/json",
      "isHarvestable": true,
      "isVisibleInUserInterface": true
    },
    "oai_ddi": {
      "displayName": "DDI",
      "mediaType": "application/xml",
      "isHarvestable": true,
      "isVisibleInUserInterface": false,
      "XMLNameSpace": "ddi:codebook:2_5",
      "XMLSchemaLocation": "https://ddialliance.org/Specification/DDI-Codebook/2.5/XMLSchema/codebook.xsd",
      "XMLSchemaVersion": "2.5"
    }
  }
}

New docs look like this:

julian-schneider · 2024-08-28T12:35:03Z

@pdurbin I believe I have addressed all the feedback you gave, have a nice day!

coveralls · 2024-08-31T11:50:45Z

coverage: 20.76% (+0.02%) from 20.741%
when pulling 3f81981 on julian-schneider:expose-export-formats
into 0d27957 on IQSS:develop.

julian-schneider added 2 commits August 2, 2024 13:50

Expose list of export formats in Info API

5d0352a

Document exportFormats method in native API guide

49dd83b

pdurbin added Size: 3 A percentage of a sprint. 2.1 hours. Type: Feature a feature request labels Aug 2, 2024

pdurbin reviewed Aug 2, 2024

View reviewed changes

julian-schneider added 2 commits August 28, 2024 10:47

Update api/info/exportFormats to give more available info, restructur…

cc99729

…ing response as JSON object

Document updated exportFormats method in native API guide

3b2c8e5

julian-schneider added 2 commits August 28, 2024 14:08

Add API test for api/info/exportFormats

95d34c8

Add release note snippet for exportFormats API method release

3f81981

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Expose export formats in native API #10739

Expose export formats in native API #10739

julian-schneider commented Aug 2, 2024

pdurbin left a comment

pdurbin Aug 2, 2024

julian-schneider Aug 28, 2024

pdurbin Aug 2, 2024

julian-schneider Aug 28, 2024

pdurbin Aug 2, 2024

julian-schneider Aug 28, 2024

pdurbin Aug 2, 2024

julian-schneider Aug 28, 2024

pdurbin Aug 2, 2024

julian-schneider Aug 28, 2024

julian-schneider commented Aug 2, 2024

julian-schneider commented Aug 28, 2024

julian-schneider commented Aug 28, 2024

coveralls commented Aug 31, 2024

		@@ -4618,6 +4618,25 @@ The fully expanded example above (without environment variables) looks like this

Expose export formats in native API #10739

Are you sure you want to change the base?

Expose export formats in native API #10739

Conversation

julian-schneider commented Aug 2, 2024

pdurbin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

julian-schneider commented Aug 2, 2024

julian-schneider commented Aug 28, 2024

julian-schneider commented Aug 28, 2024

coveralls commented Aug 31, 2024