Skip to content

Commit

Permalink
various fixes and changes #8979
Browse files Browse the repository at this point in the history
- highlight Globus and other big features at top
- added missing snippet: New Computational Workflow Metadata Block
- fix typos, etc.
  • Loading branch information
pdurbin committed Sep 29, 2022
1 parent caa43e4 commit da4d835
Showing 1 changed file with 43 additions and 23 deletions.
66 changes: 43 additions & 23 deletions doc/release-notes/5.12-release-notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,31 @@ This release brings new features, enhancements, and bug fixes to the Dataverse S

## Release Highlights

### Support for Globus

[Globus][] can be used to transfer large files. Part of "Harvard Data Commons Additions" below.

[Globus]: https://www.globus.org

### Support for Remote File Storage

Dataset files can be stored at remote URLs. Part of "Harvard Data Commons Additions" below.

### New Computational Workflow Metadata Block

The new Computational Workflow metadata block will allow depositors to effectively tag datasets as computational workflows.

To add the new metadata block, follow the instructions in the Admin Guide: <https://guides.dataverse.org/en/5.12/admin/metadatacustomization.html>

The location of the new metadata block tsv file is `scripts/api/data/metadatablocks/computational_workflow.tsv`. Part of "Harvard Data Commons Additions" below.

### Support for Linked Data Notifications (LDN)

[Linked Data Notifications][] (LDN) is a standard from the W3C. Part of "Harvard Data Commons Additions" below.

[Linked Data Notifications]: https://www.w3.org/TR/ldn/


### Harvard Data Commons Additions

As reported at the 2022 Dataverse Community Meeting, the [Harvard Data Commons](https://sites.harvard.edu/harvard-data-commons/) project has supported a wide range of additions to the Dataverse software that improve support for Big Data, Workflows, Archiving, and interaction with other repositories. In many cases, these additions build upon features developed within the Dataverse community by Borealis, DANS, QDR, TDL, and others. Highlights from this work include:
Expand All @@ -19,18 +44,14 @@ As reported at the 2022 Dataverse Community Meeting, the [Harvard Data Commons](
- Initial support for sending and receiving Linked Data Notification messages indicating relationships between a dataset and external resources (e.g. papers or other dataset) that can be used to trigger additional actions, such as the creation of a back-link to provide, for example, bi-directional linking between a published paper and a Dataverse dataset.
- A new capability to provide custom per field instructions in dataset templates
- The following file extensions are now detected:
wdl=text/x-workflow-description-language
cwl=text/x-computational-workflow-language
nf=text/x-nextflow
Rmd=text/x-r-notebook
rb=text/x-ruby-script
dag=text/x-dagman

### Update to Payara 5.2022.3 highly recommended
- wdl=text/x-workflow-description-language
- cwl=text/x-computational-workflow-language
- nf=text/x-nextflow
- Rmd=text/x-r-notebook
- rb=text/x-ruby-script
- dag=text/x-dagman

With lots of bug and security fixes included, we encourage everyone to update to Payara 5.2022.3 as soon as possible.

### Improvements to fields that appear in the Citation metadata block
### Improvements to Fields that Appear in the Citation Metadata Block

Grammar, style and consistency improvements have been made to the titles, tooltip description text, and watermarks of metadata fields that appear in the Citation metadata block.

Expand All @@ -40,24 +61,24 @@ The changes should provide clearer information to curators, depositors, and peop

A new page in the Style Guides called "Text" has also been added. The new page includes a section called "Metadata Text Guidelines" with a link to a Google Doc where the guidelines are being maintained for now since we expect them to be revised frequently.

### Adding new static search facet: Metadata Types
### New Static Search Facet: Metadata Types

A new static search facet has been added to the search side panel. This new facet is called "Metadata Types" and is driven from metadata blocks. When a metadata field value is inserted into a dataset, an entry for the metadata block it belongs to is added to this new facet.

This new facet needs to be configured for it to appear on the search side panel. The configuration assigns to a dataverse what metadata blocks to show. The configuration is inherited by child dataverses.

To configure the new facet, use the Metadata Block Facet API: <https://guides.dataverse.org/en/latest/api/native-api.html#set-metadata-block-facet-for-a-dataverse-collection>
To configure the new facet, use the Metadata Block Facet API: <https://guides.dataverse.org/en/5.12/api/native-api.html#set-metadata-block-facet-for-a-dataverse-collection>

### Broader MicroProfile Config Support for Developers

As of this release, many [JVM options](https://guides.dataverse.org/en/latest/installation/config.html#jvm-options)
As of this release, many [JVM options](https://guides.dataverse.org/en/5.12/installation/config.html#jvm-options)
can be set using any [MicroProfile Config Source](https://docs.payara.fish/community/docs/Technical%20Documentation/MicroProfile/Config/Overview.html#config-sources).

Currently this change is only relevant to developers but as settings are migrated to the new "lookup" pattern documented in the [Consuming Configuration](https://guides.dataverse.org/en/latest/developers/configuration.html) section of the Developer Guide, anyone installing the Dataverse software will have much greater flexibility when configuring those settings, especially within containers. These changes will be announced in future releases.
Currently this change is only relevant to developers but as settings are migrated to the new "lookup" pattern documented in the [Consuming Configuration](https://guides.dataverse.org/en/5.12/developers/configuration.html) section of the Developer Guide, anyone installing the Dataverse software will have much greater flexibility when configuring those settings, especially within containers. These changes will be announced in future releases.

Please note that an upgrade to Payara 5.2021.8 or higher is required to make use of this. Payara 5.2021.5 threw exceptions, as explained in PR #8823.

### HTTP Range Requests: New HTTP status codes and headers for Datafile Access API
### HTTP Range Requests: New HTTP Status Codes and Headers for Datafile Access API

The Basic File Access resource for datafiles (/api/access/datafile/$id) was slightly modified in order to comply better with the HTTP specification for range requests.

Expand All @@ -70,7 +91,7 @@ CORS rules/headers were modified accordingly:
* The "Range" header is added to "Access-Control-Allow-Headers"
* The "Content-Range" and "Accept-Ranges" header are added to "Access-Control-Expose-Headers"

### File types detection
### File Type Detection When File Has No Extension

File types are now detected based on the filename when the file has no extension.

Expand All @@ -83,6 +104,10 @@ The following filenames are now detected:

These are defined in `MimeTypeDetectionByFileName.properties`.

### Upgrade to Payara 5.2022.3 Highly Recommended

With lots of bug and security fixes included, we encourage everyone to upgrade to Payara 5.2022.3 as soon as possible. See below for details.

## Major Use Cases and Infrastructure Enhancements

Changes and fixes in this release include:
Expand Down Expand Up @@ -142,7 +167,6 @@ The Admin API call to manually submit a dataset version for archiving has change

Earlier versions of the archival bags included the ingested (tab-separated-value) version of tabular files while providing the checksum of the original file (Issue #8449). This release fixes that by including the original file and its metadata in the archival bag. This means that archival bags created prior to this version do not include a way to validate ingested files. Further, it is likely that capabilities in development (i.e. as part of the [Dataverse Uploader](https://github/org/GlobalDataverseCommunityConsortium/dataverse-uploader) to allow re-creation of a dataset version from an archival bag will only be fully compatible with archival bags generated by a Dataverse instance at a release > v5.12. (Specifically, at a minimum, since only the ingested file is included in earlier archival bags, an upload via DVUploader would not result in the same original file/ingested version as in the original dataset.) Administrators should be aware that re-creating archival bags, i.e. via the new batch archiving API, may be advisable now and will be recommended at some point in the future (i.e. there will be a point where we will start versioning archival bags and will start maintaining backward compatibility for older versions as part of transitioning this from being an experimental capability).


## Installation

If this is a new installation, please see our [Installation Guide](https://guides.dataverse.org/en/5.12/installation/). Please also contact us to get added to the [Dataverse Project Map](https://guides.dataverse.org/en/5.10/installation/config.html#putting-your-dataverse-installation-on-the-map-at-dataverse-org) if you have not done so already.
Expand All @@ -155,10 +179,7 @@ If you are running Payara as a non-root user (and you should be!), **remember no

In the following commands we assume that Payara 5 is installed in `/usr/local/payara5`. If not, adjust as needed.


# Update to Payara 5.2022.3 highly recommended

With lots of bug and security fixes included, we encourage everyone to update to Payara 5.2022.3 as soon as possible.
### Instructions for Upgrading to Payara 5.2022.3

**Note:** with the approaching EOL for the Payara 5 Community release train it's likely we will switch to a
yet-to-be-released Payara 6 in the not-so-far-away future.
Expand All @@ -172,7 +193,6 @@ The steps below are a simple matter of reusing your existing domain directory wi
But we also recommend that you review the Payara upgrade instructions as it could be helpful during any troubleshooting:
[Payara Release Notes](https://docs.payara.fish/community/docs/Release%20Notes/Release%20Notes%205.2022.3.html)


Please note that the deletion of the `lib/databases` directory below is only required once, for this upgrade (see Issue #8230 for details).

```shell
Expand Down

0 comments on commit da4d835

Please sign in to comment.