Plugin mechanism to support further PID providers

[fbgesis]

Different institutions have different needs and preferences for dataset registration agencies/PID providers they want cooperate with. Thus, it would be nice to use DV with the registration agency of one's choice.

We propose a framework based on Java Service Loader to extend DV with functionality required to communicate with additional registration agencies.

• The interaction can be based on events e.g. onCreate, onUpdate, onPublish,... so every provider can use its own workflow.

• The plugin collects required metadata from the host application, contacts the registration agency and returns a DOI or Handle.

• A sophisticated set of Java interfaces decouples DV and plugins and regulates interoperability.

• A more detailed set of exceptions enables specific error handling.

• ...

An early design can be found here:
https://git.gesis.org/dsn/dv-pid-plugin/tree/master

[djbrooke]

Thanks @fbgesis ! Good to talk with you this morning, and thanks for creating this issue.

As we mentioned, @scolapasta is taking a look at this and will pull in other developers as needed.

[kaczmirek]

@fbgesis your initiative is much appreciated. Here at AUSSDA - The Austrian Social Science Data Archive we are in urgent need for an easy way to get our DOIs automatically delivered with our own prefix. I hope this will be on the official roadmap soon. Especially, so that we have this implemented before end 2018 as this will be an important achievement that we want to report to our external evaluators. Thanks again!

[pdurbin]

Yes, it's awesome that @fbgesis is taking this on. Thanks!

@kaczmirek I don't expect you'll see this issue on the official roadmap at https://dataverse.org/goals-roadmap-and-releases but this project has been added to"Dev Efforts by the Dataverse Community" spreadsheet: https://docs.google.com/spreadsheets/d/1pl9U0_CtWQ3oz6ZllvSHeyB0EG1M_vZEC_aZ7hREnhE/edit?usp=sharing as discussed during the 2017-08-29 Dataverse Community Call yesterday: https://groups.google.com/d/msg/dataverse-community/EYjhYIVVvRU/wtCpI-uMAQAJ

@fbgesis I was just looking at https://git.gesis.org/dsn/dv-pid-plugin/blob/ec84bc651f1fcfc18c98716ed6cfa9e3de5ab0e4/plugin-test-container/src/main/java/edu/harvard/iq/dataverse/pidplugin/IPIDProvider.java#L58 and wondering how the URL for the PID provider would be configured. As of this writing, the DOI provider URLs are either EZID ( https://ezid.cdlib.org ) or DataCite ( https://mds.datacite.org or https://researchdata.ands.org.au/api/doi/datacite ). I'm curious about what sort of DOI provider URLs will be used with this plugin mechanism. Are these DOI provider URLs (services) hosted by the institution that installed Dataverse? Are the URLs (services) hosted centrally by a government agency within a country?

[fbgesis]

@pdurbin I thougth we could stick to the current way and use Java properties e.g. http://guides.dataverse.org/en/latest/installation/config.html#doi-baseurlstring but it could also be set via the DV API and then forwarded to the plugin by methods that we will then have to add.

Your second question: In general those registration agencies can be third party institutions of any kind, just like EZID and DataCite. Currently those are hardwired, in the future they might be available as the two standard PID plugins. In addidtion to this we will provide a DV da|ra PID plugin which allows for registration with da|ra https://www.da-ra.de/en/home/ .

Afaik @4tikhonov from DANS might also have to say sth. to this.

[lmaylein]

heiDATA (Heidelberg Research Data Repository) is strongly interested in a da|ra PID plugin.

[pdurbin]

@lmaylein good to know that multiple installations of Dataverse can benefit from this DOI provider.

@fbgesis If it's quicker or easier, I wouldn't object to there being a pull request that simply implements the existing AbstractIdServiceBean: https://github.com/IQSS/dataverse/blob/v4.7.1/src/main/java/edu/harvard/iq/dataverse/AbstractIdServiceBean.java . We could always refactor it later into more of a plugin architecture. What's more important is getting the code working and providing value.

[pdurbin]

@fbgesis hi! I was just updating the "Dev Efforts by the Dataverse Community" spreadsheet at https://docs.google.com/spreadsheets/d/1pl9U0_CtWQ3oz6ZllvSHeyB0EG1M_vZEC_aZ7hREnhE/edit?usp=sharing and noticed that https://git.gesis.org/dsn/dv-pid-plugin says " Archived project! Repository is read-only". Is https://git.gesis.org/dsn/dara-dv-plugin the location of the code? Any other updates on this project? Thanks!

[fbgesis]

Hi @pdurbin, these repos were meant to illustrate our ideas, we keep them online for ducumentary purporses. Meanwhile we integrated the plungin loader mechanism into the DataverseEU project.

There is a prototype with PID/da|ra support and other new features available at https://dataverse-dev.cessda.eu/ . It is currently tested by service providers (Social Science Institutions) from Europe.
@4tikhonov should be able to help you with up-to-date code.
However, we are still interested in seeing the mechanism in the Dataverse master branch.

[pdurbin]

@fbgesis thanks. @4tikhonov any comments? How do we move this issue forward?

[4tikhonov]

We have a meeting in Cologne to discuss this issue with @fbgesis in two weeks. We'll try to change the architecture to get this plugin working outside of Dataverse core to avoid all maintenance problems.

[poikilotherm]

Hi @4tikhonov and @fbgesis,

I'm working at Forschungszentrum Jülichs RDM team and we are evaluating to use Dataverse as a RD repo.

We would like to use ePIC PIDs for our datasets instead or in combination with Dataverse DOIs.

There are good chances that we at FZJ would invest man power to implement the provider (most certainly me). Maybe I could learn from your experience with the da|ra provider for an implementation?

Thx
Oliver

[scolapasta]

So recently we've worked on a lot of code that has touched these areas, File DOIs, #3083. Once that last one gets merged, I think it would be the right time to get the work that @fbgesis did merged with the core and then we could also help work with you for ePic PIDs.

[poikilotherm]

@scolapasta sound great! 😄

[4tikhonov]

Great @scolapasta, if you'll be fast enough then probably we can cancel our meeting in Cologne. :)

[scolapasta]

I thought you were going to say you were inviting me there! :)

Seriously, though, #3083 is in QA right now, so hopefully it'll be merged soon, and this is on my list to address as soon as that happens.

[poikilotherm]

Hi @4tikhonov and @fbgesis any news on this? Can I help out or otherwise participate in any way?

[pdurbin]

Look out, @poikilotherm is already making pull requests for other issues! See pull request #5061 for example. 😄 Can we put him to work on this issue? 😄

[poikilotherm]

Notes to self:

• some examples for plugin mechanisms https://www.phillip-kruger.com/post/some_factory_examples/
• a very enlightning talk by @4tikhonov https://slideshare.net/vty/persistent-identifiers-in-dataverseeu-project

[pdurbin]

I just noticed "Dataverse 4.9.3 with da-ra support" at https://github.com/IQSS/dataverse-docker/releases/tag/4.9.3

[4tikhonov]

Yes, we're working with GESIS on da-ra plugin where plugin is external application (with own Docker image) that consumes external mappings. It will be universal and will allow to connect Dataverse to any other system.

[pdurbin]

@4tikhonov great! Did you have to fork Dataverse to get da-ra support?

[4tikhonov]

@pdurbin, we're trying to made just as slight changes as possible to get it merged with master branch in near future. Also we've modified frontend to get support of CESSDA controlled vocabularies coming from API endpoint delivered by GESIS.

[pdurbin]

@4tikhonov you know we love slight changes. We often refer to "small chunks" in our sprint planning meetings. Please keep us posted! Thanks! #5327 is potentially related.

[pdurbin]

@4tikhonov Happy New Year! Is this issue blocked? Do you need anything? Heads up that we did merge a pull request for #5327 and there's a new pull request in flight for #5427.

[pdurbin]

On Wednesday "mouse-nioz" asked, "Is it possible to change the PID Provider in dataverse?" and I pointed them to this issue. Here was the short conversation: http://irclog.iq.harvard.edu/dataverse/2019-03-13#i_87985

The use case was "for testing purposes we want to use the "handle.test.datacite.org" as provider. now from what I understand our dataverse tries to use "doi.org" which we don't register to in our test cases".

[poikilotherm]

IMHO this issue can be closed. We will try to use other approaches for a solution to this.
Current ideas are about creating a separate web service "speaking" DataCite API for a start, which would take load off from Dataverse core devs.
AFAIK DANS (especially @4tikhonov) implemented some special service for their dara/DataCite problem.

[pdurbin]

I'm un-assigning @poikilotherm based on his comment above and chatter at http://irclog.iq.harvard.edu/dataverse/2019-03-27

@4tikhonov what is your status, please? Do you still want or need this? Should you still be assigned?

[4tikhonov]

@pdurbin, yes, we'll continue the plugin development in the new SSHOC DataverseEU project with CLARIN and DARIAH people onboard.

[pdurbin]

@4tikhonov did you give an update on this issue in a GitHub comment a week or two ago? I can't find it. Maybe it was only a dream! 😄

[pdurbin]

Related:

• Support flexible option for DOI and Handle registration, in same installation #3623
• Beyond Dataverse 5.0: make it pluggable (on a code level) #7050
• Support Archival Resource Key (ARK) as Persistent Identifier #7705
• Adding Crossref as a DOI provider  #8581

[cmbz]

2024/09/09: Closing. If folks need additional functionality, please create a new issue.

[pdurbin]

Yes, #10234 gets us much closer.