From e84671129e2191c57e1ae9274388cfe8ca8551f2 Mon Sep 17 00:00:00 2001 From: Danny Brooke Date: Thu, 22 Jul 2021 13:39:05 -0400 Subject: [PATCH 01/11] out with the old... --- doc/release-notes/5.6-release-notes.md | 71 ++++++++++++++++++++++++++ 1 file changed, 71 insertions(+) create mode 100644 doc/release-notes/5.6-release-notes.md diff --git a/doc/release-notes/5.6-release-notes.md b/doc/release-notes/5.6-release-notes.md new file mode 100644 index 00000000000..663bcc29f2c --- /dev/null +++ b/doc/release-notes/5.6-release-notes.md @@ -0,0 +1,71 @@ +# Dataverse Software 5.5 + +This release brings new features, enhancements, and bug fixes to the Dataverse Software. Thank you to all of the community members who contributed code, suggestions, bug reports, and other assistance across the project. + +## Release Highlights + +### XXX + +XXX + +## Major Use Cases + +Newly-supported major use cases in this release include: + +- XX (Issue #XXX, PR #XXX) + +## Notes for Dataverse Installation Administrators + +### XXX + +XXX + +## Notes for Tool Developers and Integrators + +### XXX + +## Complete List of Changes + +For the complete list of code changes in this release, see the [5.6 Milestone](https://github.com/IQSS/dataverse/milestone/97?closed=1) in Github. + +For help with upgrading, installing, or general questions please post to the [Dataverse Community Google Group](https://groups.google.com/forum/#!forum/dataverse-community) or email support@dataverse.org. + +## Installation + +If this is a new installation, please see our [Installation Guide](https://guides.dataverse.org/en/5.6/installation/). + +## Upgrade Instructions + +0\. These instructions assume that you've already successfully upgraded from Dataverse Software 4.x to Dataverse Software 5 following the instructions in the [Dataverse Software 5 Release Notes](https://github.com/IQSS/dataverse/releases/tag/v5.0). After upgrading from the 4.x series to 5.0, you should progress through the other 5.x releases before attempting the upgrade to 5.6. + +1\. Undeploy the previous version. + +- `$PAYARA/bin/asadmin list-applications` +- `$PAYARA/bin/asadmin undeploy dataverse<-version>` + +2\. Stop Payara and remove the generated directory + +- `service payara stop` +- `rm -rf $PAYARA/glassfish/domains/domain1/generated` + +3\. Start Payara + +- `service payara start` + +4\. Deploy this version. + +- `$PAYARA/bin/asadmin deploy dataverse-5.6.war` + +5\. Restart payara + +- `service payara stop` +- `service payara start` + +## Additional Release Steps + +1\. Follow the steps to update your Solr configuration, found in the "Notes for Dataverse Installation Administrators" section above. Note that there are different instructions for Dataverse installations running with custom metadata blocks and those without. + +2\. Update Geospatial Metadata Block (if used) + +- `wget https://github.com/IQSS/dataverse/releases/download/v5.5/geospatial.tsv` +- `curl http://localhost:8080/api/admin/datasetfield/load -X POST --data-binary @geospatial.tsv -H "Content-type: text/tab-separated-values"` From 26de08901cdab6cc56814cc7a27857ec579dabf0 Mon Sep 17 00:00:00 2001 From: Danny Brooke Date: Thu, 22 Jul 2021 13:59:40 -0400 Subject: [PATCH 02/11] in with the new... --- doc/release-notes/5.6-release-notes.md | 52 ++++++++++++++++--- doc/release-notes/6497-semantic-api.md | 7 --- .../7548-stored-procedure-update.md | 15 ------ doc/release-notes/7700-upgrade-payara.md | 19 ------- doc/release-notes/7786-ddi_changes.md | 4 +- .../7900-add-multipleFilesMetadata-dataset.md | 4 -- 6 files changed, 46 insertions(+), 55 deletions(-) delete mode 100644 doc/release-notes/6497-semantic-api.md delete mode 100644 doc/release-notes/7548-stored-procedure-update.md delete mode 100644 doc/release-notes/7700-upgrade-payara.md delete mode 100644 doc/release-notes/7900-add-multipleFilesMetadata-dataset.md diff --git a/doc/release-notes/5.6-release-notes.md b/doc/release-notes/5.6-release-notes.md index 663bcc29f2c..afb8768932b 100644 --- a/doc/release-notes/5.6-release-notes.md +++ b/doc/release-notes/5.6-release-notes.md @@ -4,9 +4,16 @@ This release brings new features, enhancements, and bug fixes to the Dataverse S ## Release Highlights -### XXX +### Dataset Semantic API (Experimental) + +Dataset metadata can be retrieved/set/updated using a new, flatter JSON-LD format - following the format of an OAI-ORE export (RDA-conformant Bags), allowing for easier transfer of metadata to/from other systems (i.e. without needing to know Dataverse's metadata block and field storage architecture). This new API also allows for the update of terms metadata (#5899). + +This development was supported by the [Research Data Alliance](https://rd-alliance.org), DANS, and Sciences PO and follows the recommendations from the [Research Data Repository Interoperability Working Group](http://dx.doi.org/10.15497/RDA00025). -XXX +### Direct Upload API Now Available for Adding Multiple Files Metadata to the Dataset + +Users can now add metadata of multiple files to the dataset once the files exists in the s3 bucket using the direct upload API. +For more information, see the [Direct DataFile Upload/Replace API section](https://guides.dataverse.org/en/5.6/developers/s3-direct-upload-api.html) of the Dataverse Software Guides. ## Major Use Cases @@ -16,9 +23,31 @@ Newly-supported major use cases in this release include: ## Notes for Dataverse Installation Administrators -### XXX +### Payara 5.2021.4 (or Higher) Required + +Some changes in this release require an upgrade to Payara 5.2021.4 or higher. + +Instructions on how to update can be found in the +[Payara documentation](https://docs.payara.fish/community/docs/5.2021.4/documentation/user-guides/upgrade-payara.html) + +It would likely be safer to upgrade Payara first, while still running Dataverse 5.6, and then proceed with the steps +below. Upgrading from an earlier version of Payara should be a straightforward process: + +1. Undeploy Dataverse +2. Stop Payara +3. Move the current Payara directory out of the way +4. Unzip the new Payara version in its place +5. Replace the brand new payara/glassfish/domains/domain1 with your old, preserved `domain1` +6. Start Payara, deploy Dataverse 5.6. + +We still recommend that you read the detailed upgrade instructions above and if you run into any issues with this +upgrade, it will help to be able to separate them from any problems with the upgrade of Dataverse proper. + +### Enhancement to DDI Metadata Exports + +Several changes have been made to the DDI exports to improve support for internationalization and to improve compliance with CESSDA requirements. These changes include: -XXX +* Addition of a holdings element with a URI attribute whose value is the URL form of the dataset PID ## Notes for Tool Developers and Integrators @@ -63,9 +92,16 @@ If this is a new installation, please see our [Installation Guide](https://guide ## Additional Release Steps -1\. Follow the steps to update your Solr configuration, found in the "Notes for Dataverse Installation Administrators" section above. Note that there are different instructions for Dataverse installations running with custom metadata blocks and those without. +**If your installation relies on the database-side stored procedure for generating sequential numeric identifiers:** -2\. Update Geospatial Metadata Block (if used) +*(Note: You can skip the following paragraph if your installation uses the default-style, randomly-generated six alphanumeric +character-long identifiers for your datasets!)* -- `wget https://github.com/IQSS/dataverse/releases/download/v5.5/geospatial.tsv` -- `curl http://localhost:8080/api/admin/datasetfield/load -X POST --data-binary @geospatial.tsv -H "Content-type: text/tab-separated-values"` +The underlying database framework has been modified in this release, to make it easier for installations +to create custom procedures for generating identifier strings that suit their needs. Your current configuration will +be automatically updated by the database upgrade (Flyway) script incorporated in the release. No manual configuration +changes should be necessary. However, after the upgrade, we recommend that you confirm that your installation can still +create new datasets, and that they are still assigned sequential numeric identifiers. In the unlikely chance that this +is no longer working, please re-create the stored procedure following the steps described in the documentation for the +`:IdentifierGenerationStyle` setting in the *Configuration* section of the Installation Guide for this release (v5.6). +(Running the script supplied there will NOT overwrite the position on the sequence you are currently using!) diff --git a/doc/release-notes/6497-semantic-api.md b/doc/release-notes/6497-semantic-api.md deleted file mode 100644 index 8222892d18e..00000000000 --- a/doc/release-notes/6497-semantic-api.md +++ /dev/null @@ -1,7 +0,0 @@ -# Release Highlights - -### Dataset Semantic API (Experimental) - -Dataset metadata can be retrieved/set/updated using a new, flatter JSON-LD format - following the format of an OAI-ORE export (RDA-conformant Bags), allowing for easier transfer of metadata to/from other systems (i.e. without needing to know Dataverse's metadata block and field storage architecture). This new API also allows for the update of terms metadata (#5899). - -This development was supported by the [Research Data Alliance](https://rd-alliance.org), DANS, and Sciences PO and follows the recommendations from the [Research Data Repository Interoperability Working Group](http://dx.doi.org/10.15497/RDA00025). diff --git a/doc/release-notes/7548-stored-procedure-update.md b/doc/release-notes/7548-stored-procedure-update.md deleted file mode 100644 index 26d2a8ab784..00000000000 --- a/doc/release-notes/7548-stored-procedure-update.md +++ /dev/null @@ -1,15 +0,0 @@ -### Upgrade Notes - -**If your installation relies on the database-side stored procedure for generating sequential numeric identifiers:** - -*(Note: You can skip the following paragraph if your installation uses the default-style, randomly-generated six alphanumeric -character-long identifiers for your datasets!)* - -The underlying database framework has been modified in this release, to make it easier for installations -to create custom procedures for generating identifier strings that suit their needs. Your current configuration will -be automatically updated by the database upgrade (Flyway) script incorporated in the release. No manual configuration -changes should be necessary. However, after the upgrade, we recommend that you confirm that your installation can still -create new datasets, and that they are still assigned sequential numeric identifiers. In the unlikely chance that this -is no longer working, please re-create the stored procedure following the steps described in the documentation for the -`:IdentifierGenerationStyle` setting in the *Configuration* section of the Installation Guide for this release (v5.6). -(Running the script supplied there will NOT overwrite the position on the sequence you are currently using!) diff --git a/doc/release-notes/7700-upgrade-payara.md b/doc/release-notes/7700-upgrade-payara.md deleted file mode 100644 index de7c58bf963..00000000000 --- a/doc/release-notes/7700-upgrade-payara.md +++ /dev/null @@ -1,19 +0,0 @@ -### Payara 5.2021.4 (or Higher) Required - -Some changes in this release require an upgrade to Payara 5.2021.4 or higher. - -Instructions on how to update can be found in the -[Payara documentation](https://docs.payara.fish/community/docs/5.2021.4/documentation/user-guides/upgrade-payara.html) - -It would likely be safer to upgrade Payara first, while still running Dataverse 5.6, and then proceed with the steps -below. Upgrading from an earlier version of Payara should be a straightforward process: - -1. Undeploy Dataverse -2. Stop Payara -3. Move the current Payara directory out of the way -4. Unzip the new Payara version in its place -5. Replace the brand new payara/glassfish/domains/domain1 with your old, preserved `domain1` -6. Start Payara, deploy Dataverse 5.6. - -We still recommend that you read the detailed upgrade instructions above and if you run into any issues with this -upgrade, it will help to be able to separate them from any problems with the upgrade of Dataverse proper. diff --git a/doc/release-notes/7786-ddi_changes.md b/doc/release-notes/7786-ddi_changes.md index c6b0d455ddf..6cbc1cfb17c 100644 --- a/doc/release-notes/7786-ddi_changes.md +++ b/doc/release-notes/7786-ddi_changes.md @@ -1,9 +1,9 @@ -Note: These notes cover several related PRs (#7984, #7958, #7959 respectively for the three bullets below.) If some are not merged before the next release, these notes should be adjusted. +Note: These notes cover several related PRs (#7958, #7959 respectively for the three bullets below.) If some are not merged before the next release, these notes should be adjusted. ### Enhancements to DDI Metadata Exports Several changes have been made to the DDI exports to improve support for internationalization and to improve compliance with CESSDA requirements. These changes include: -* Addition of a holdings element with a URI attribute whose value is the URL form of the dataset PID + * Addition of xml:lang attributes specifying the dataset metadata language at the document level and for individual elements such as title and description * Specification of controlled vocabulary terms in duplicate elements in multiple languages (in the installation default langauge and, if different, the dataset metadata language) diff --git a/doc/release-notes/7900-add-multipleFilesMetadata-dataset.md b/doc/release-notes/7900-add-multipleFilesMetadata-dataset.md deleted file mode 100644 index 13b77259c69..00000000000 --- a/doc/release-notes/7900-add-multipleFilesMetadata-dataset.md +++ /dev/null @@ -1,4 +0,0 @@ -### Direct Upload API Now Available for Adding Multiple Files Metadata to the Dataset - -Users can now add metadata of multiple files to the dataset once the files exists in the s3 bucket using the direct upload API. -For more information, see the [Direct DataFile Upload/Replace API section](https://guides.dataverse.org/en/5.6/developers/s3-direct-upload-api.html) of the Dataverse Software Guides. From 80a92c5bacef1a0d84be2b429f105fa6780699c9 Mon Sep 17 00:00:00 2001 From: Danny Brooke Date: Thu, 22 Jul 2021 15:24:15 -0400 Subject: [PATCH 03/11] some stubs for stuff that I feel raises to level of release highlights --- doc/release-notes/5.6-release-notes.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/doc/release-notes/5.6-release-notes.md b/doc/release-notes/5.6-release-notes.md index afb8768932b..9b94f724134 100644 --- a/doc/release-notes/5.6-release-notes.md +++ b/doc/release-notes/5.6-release-notes.md @@ -1,9 +1,15 @@ -# Dataverse Software 5.5 +# Dataverse Software 5.6 This release brings new features, enhancements, and bug fixes to the Dataverse Software. Thank you to all of the community members who contributed code, suggestions, bug reports, and other assistance across the project. ## Release Highlights +### Anonymized Accesses in Support of Double Blind Review + +### Guestbooks API + + + ### Dataset Semantic API (Experimental) Dataset metadata can be retrieved/set/updated using a new, flatter JSON-LD format - following the format of an OAI-ORE export (RDA-conformant Bags), allowing for easier transfer of metadata to/from other systems (i.e. without needing to know Dataverse's metadata block and field storage architecture). This new API also allows for the update of terms metadata (#5899). @@ -15,6 +21,7 @@ This development was supported by the [Research Data Alliance](https://rd-allian Users can now add metadata of multiple files to the dataset once the files exists in the s3 bucket using the direct upload API. For more information, see the [Direct DataFile Upload/Replace API section](https://guides.dataverse.org/en/5.6/developers/s3-direct-upload-api.html) of the Dataverse Software Guides. + ## Major Use Cases Newly-supported major use cases in this release include: From 9ef174db163bc7659699f5901d15da82a53607cd Mon Sep 17 00:00:00 2001 From: Danny Brooke Date: Thu, 22 Jul 2021 17:49:43 -0400 Subject: [PATCH 04/11] additional updates, will get back to it tomorrow --- doc/release-notes/5.6-release-notes.md | 29 +++++++++++++++++++------- 1 file changed, 21 insertions(+), 8 deletions(-) diff --git a/doc/release-notes/5.6-release-notes.md b/doc/release-notes/5.6-release-notes.md index 9b94f724134..36921c620d3 100644 --- a/doc/release-notes/5.6-release-notes.md +++ b/doc/release-notes/5.6-release-notes.md @@ -6,30 +6,39 @@ This release brings new features, enhancements, and bug fixes to the Dataverse S ### Anonymized Accesses in Support of Double Blind Review -### Guestbooks API +Dataverse installations can select whether or not to allow users to create anonymized private URLs and can control which specific fields are anonymized. +If this is enabled at the installation level, users can create private URLs that do not expose identifying information about dataset depositors, allowing for double blind reviews of datasets. +### Guestbook Responses API + +A new API to retrieve Guestbook responses has been added. This makes it easier to retrieve the records for large guestbooks and also makes it easier to integrate with external systems. ### Dataset Semantic API (Experimental) -Dataset metadata can be retrieved/set/updated using a new, flatter JSON-LD format - following the format of an OAI-ORE export (RDA-conformant Bags), allowing for easier transfer of metadata to/from other systems (i.e. without needing to know Dataverse's metadata block and field storage architecture). This new API also allows for the update of terms metadata (#5899). +Dataset metadata can be retrieved, set, and updated using a new, flatter JSON-LD format - following the format of an OAI-ORE export (RDA-conformant Bags), allowing for easier transfer of metadata to/from other systems (i.e. without needing to know Dataverse's metadata block and field storage architecture). This new API also allows for the update of terms metadata (#5899). This development was supported by the [Research Data Alliance](https://rd-alliance.org), DANS, and Sciences PO and follows the recommendations from the [Research Data Repository Interoperability Working Group](http://dx.doi.org/10.15497/RDA00025). ### Direct Upload API Now Available for Adding Multiple Files Metadata to the Dataset -Users can now add metadata of multiple files to the dataset once the files exists in the s3 bucket using the direct upload API. -For more information, see the [Direct DataFile Upload/Replace API section](https://guides.dataverse.org/en/5.6/developers/s3-direct-upload-api.html) of the Dataverse Software Guides. - +Users can now add metadata of multiple files to the dataset once the files exists in the s3 bucket using the direct upload API. For more information, see the [Direct DataFile Upload/Replace API section](https://guides.dataverse.org/en/5.6/developers/s3-direct-upload-api.html) of the Dataverse Software Guides. ## Major Use Cases Newly-supported major use cases in this release include: -- XX (Issue #XXX, PR #XXX) +- Double blind (Issue #XXX, PR #XXX) +- Guestbook API (Issue #XXX, PR #XXX) +- Semantic API (Issue #XXX, PR #XXX) +- Extended Traces API (Issue #XXX, PR #XXX) +- Terms API updates (Issue #XXX, PR #XXX) ## Notes for Dataverse Installation Administrators +### DB Constraint + + ### Payara 5.2021.4 (or Higher) Required Some changes in this release require an upgrade to Payara 5.2021.4 or higher. @@ -38,7 +47,7 @@ Instructions on how to update can be found in the [Payara documentation](https://docs.payara.fish/community/docs/5.2021.4/documentation/user-guides/upgrade-payara.html) It would likely be safer to upgrade Payara first, while still running Dataverse 5.6, and then proceed with the steps -below. Upgrading from an earlier version of Payara should be a straightforward process: +below. Upgrading from an earlier version of Payara should be a straightforward process: 1. Undeploy Dataverse 2. Stop Payara @@ -46,7 +55,7 @@ below. Upgrading from an earlier version of Payara should be a straightforward p 4. Unzip the new Payara version in its place 5. Replace the brand new payara/glassfish/domains/domain1 with your old, preserved `domain1` 6. Start Payara, deploy Dataverse 5.6. - + We still recommend that you read the detailed upgrade instructions above and if you run into any issues with this upgrade, it will help to be able to separate them from any problems with the upgrade of Dataverse proper. @@ -56,6 +65,10 @@ Several changes have been made to the DDI exports to improve support for interna * Addition of a holdings element with a URI attribute whose value is the URL form of the dataset PID +## New JVM Options and DB Settings + +:AnonymizedFieldTypeNames + ## Notes for Tool Developers and Integrators ### XXX From 55463f9d8f23c6344aae3d07d1672a944f9477c3 Mon Sep 17 00:00:00 2001 From: Danny Brooke Date: Fri, 23 Jul 2021 19:11:36 -0400 Subject: [PATCH 05/11] updates before PR --- doc/release-notes/5.6-release-notes.md | 69 ++++++++++---------------- 1 file changed, 27 insertions(+), 42 deletions(-) diff --git a/doc/release-notes/5.6-release-notes.md b/doc/release-notes/5.6-release-notes.md index 36921c620d3..920c37c26d3 100644 --- a/doc/release-notes/5.6-release-notes.md +++ b/doc/release-notes/5.6-release-notes.md @@ -4,11 +4,9 @@ This release brings new features, enhancements, and bug fixes to the Dataverse S ## Release Highlights -### Anonymized Accesses in Support of Double Blind Review +### Anonymized Access in Support of Double Blind Review -Dataverse installations can select whether or not to allow users to create anonymized private URLs and can control which specific fields are anonymized. - -If this is enabled at the installation level, users can create private URLs that do not expose identifying information about dataset depositors, allowing for double blind reviews of datasets. +Dataverse installations can select whether or not to allow users to create anonymized private URLs and can control which specific identifying fields are anonymized. If this is enabled, users can create private URLs that do not expose identifying information about dataset depositors, allowing for double blind reviews of datasets in the Dataverse installation. ### Guestbook Responses API @@ -28,50 +26,37 @@ Users can now add metadata of multiple files to the dataset once the files exist Newly-supported major use cases in this release include: -- Double blind (Issue #XXX, PR #XXX) -- Guestbook API (Issue #XXX, PR #XXX) -- Semantic API (Issue #XXX, PR #XXX) -- Extended Traces API (Issue #XXX, PR #XXX) -- Terms API updates (Issue #XXX, PR #XXX) +- Users can create Private URLs that anonymize dataset metadata, allowing for double blind peer review. (Issue #1724, PR #7908) +- Users can download Guestbook records using a new API. (Issue #7767, PR #7931) +- Users can update terms metadata using the new semantic API. (Issue #5899, PR #7414) +- Users can retrieve, set, and update metadata using a new, flatter JSON-LD format. (Issue #6497, PR #7414) +- Administrators can use the Traces API to retreive information about specific types of user activity (Issue #7952, PR #7953) ## Notes for Dataverse Installation Administrators -### DB Constraint +### New Database Constraint +A new DB Constraint has been added in this release. Full instructions on how to identify whether or not your database needs any cleanup before the upgrade can be found in the [Dataverse software GitHub repository](https://github.com/IQSS/dataverse/blob/develop/scripts/issues/7451/PRE-RELEASE-INFO.txt). This information was also emailed out to Dataverse installation contacts. ### Payara 5.2021.4 (or Higher) Required Some changes in this release require an upgrade to Payara 5.2021.4 or higher. -Instructions on how to update can be found in the -[Payara documentation](https://docs.payara.fish/community/docs/5.2021.4/documentation/user-guides/upgrade-payara.html) - -It would likely be safer to upgrade Payara first, while still running Dataverse 5.6, and then proceed with the steps -below. Upgrading from an earlier version of Payara should be a straightforward process: - -1. Undeploy Dataverse -2. Stop Payara -3. Move the current Payara directory out of the way -4. Unzip the new Payara version in its place -5. Replace the brand new payara/glassfish/domains/domain1 with your old, preserved `domain1` -6. Start Payara, deploy Dataverse 5.6. - -We still recommend that you read the detailed upgrade instructions above and if you run into any issues with this -upgrade, it will help to be able to separate them from any problems with the upgrade of Dataverse proper. +Instructions on how to update can be found in the [Payara documentation](https://docs.payara.fish/community/docs/5.2021.4/documentation/user-guides/upgrade-payara.html) We've included the necessary steps below, but we recommend that you review the Payara upgrade instructions as it could be helpful during any troubleshooting. ### Enhancement to DDI Metadata Exports -Several changes have been made to the DDI exports to improve support for internationalization and to improve compliance with CESSDA requirements. These changes include: - -* Addition of a holdings element with a URI attribute whose value is the URL form of the dataset PID +To increase support for internationalization and to improve compliance with CESSDA requirements, DDI exports now have a holdings element with a URI attribute whose value is the URL form of the dataset PID. ## New JVM Options and DB Settings -:AnonymizedFieldTypeNames +:AnonymizedFieldTypeNames can be used to identify specific fields that will be anonymized when a user creates an anonymized Private URL. ## Notes for Tool Developers and Integrators -### XXX +### Semantic API + +The new Semantic API is especially helpful in data migrations and getting metadata into a Dataverse installation. Learn more in the [Developers Guide](https://guides.dataverse.org/en/5.6/developers/). ## Complete List of Changes @@ -97,15 +82,21 @@ If this is a new installation, please see our [Installation Guide](https://guide - `service payara stop` - `rm -rf $PAYARA/glassfish/domains/domain1/generated` -3\. Start Payara +3\. Move the current Payara directory out of the way + +4\. Unzip the new Payara version in its place + +5\. Replace the brand new payara/glassfish/domains/domain1 with your old, preserved `domain1` + +6\. Start Payara - `service payara start` -4\. Deploy this version. +7\. Deploy this version. - `$PAYARA/bin/asadmin deploy dataverse-5.6.war` -5\. Restart payara +8\. Restart payara - `service payara stop` - `service payara start` @@ -114,14 +105,8 @@ If this is a new installation, please see our [Installation Guide](https://guide **If your installation relies on the database-side stored procedure for generating sequential numeric identifiers:** -*(Note: You can skip the following paragraph if your installation uses the default-style, randomly-generated six alphanumeric -character-long identifiers for your datasets!)* +Note that you can skip this step if your installation uses the default-style, randomly-generated six alphanumeric character-long identifiers for your datasets! This is the case with most Dataverse installations. + +The underlying database framework has been modified in this release, to make it easier for installations to create custom procedures for generating identifier strings that suit their needs. Your current configuration will be automatically updated by the database upgrade (Flyway) script incorporated in the release. No manual configuration changes should be necessary. However, after the upgrade, we recommend that you confirm that your installation can still create new datasets, and that they are still assigned sequential numeric identifiers. In the unlikely chance that this is no longer working, please re-create the stored procedure following the steps described in the documentation for the `:IdentifierGenerationStyle` setting in the *Configuration* section of the Installation Guide for this release (v5.6). -The underlying database framework has been modified in this release, to make it easier for installations -to create custom procedures for generating identifier strings that suit their needs. Your current configuration will -be automatically updated by the database upgrade (Flyway) script incorporated in the release. No manual configuration -changes should be necessary. However, after the upgrade, we recommend that you confirm that your installation can still -create new datasets, and that they are still assigned sequential numeric identifiers. In the unlikely chance that this -is no longer working, please re-create the stored procedure following the steps described in the documentation for the -`:IdentifierGenerationStyle` setting in the *Configuration* section of the Installation Guide for this release (v5.6). (Running the script supplied there will NOT overwrite the position on the sequence you are currently using!) From 74ba6b69e59c9e66678fb3ba9f70ee309eeaaaab Mon Sep 17 00:00:00 2001 From: Danny Brooke Date: Tue, 27 Jul 2021 10:18:46 -0400 Subject: [PATCH 06/11] updates from CR --- doc/release-notes/5.6-release-notes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/release-notes/5.6-release-notes.md b/doc/release-notes/5.6-release-notes.md index 920c37c26d3..26fcb373689 100644 --- a/doc/release-notes/5.6-release-notes.md +++ b/doc/release-notes/5.6-release-notes.md @@ -20,7 +20,7 @@ This development was supported by the [Research Data Alliance](https://rd-allian ### Direct Upload API Now Available for Adding Multiple Files Metadata to the Dataset -Users can now add metadata of multiple files to the dataset once the files exists in the s3 bucket using the direct upload API. For more information, see the [Direct DataFile Upload/Replace API section](https://guides.dataverse.org/en/5.6/developers/s3-direct-upload-api.html) of the Dataverse Software Guides. +Users can now add metadata of multiple files to the dataset once the files exists in the s3 bucket using the direct upload API. This makes direct uploads more efficient and reduces server load by only updating the dataset once instead of once per file. For more information, see the [Direct DataFile Upload/Replace API section](https://guides.dataverse.org/en/5.6/developers/s3-direct-upload-api.html) of the Dataverse Software Guides. ## Major Use Cases From 669adb3022c091b69811e8f903edb095cc1342c6 Mon Sep 17 00:00:00 2001 From: Danny Brooke Date: Tue, 27 Jul 2021 10:19:46 -0400 Subject: [PATCH 07/11] updates from CR --- doc/release-notes/5.6-release-notes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/release-notes/5.6-release-notes.md b/doc/release-notes/5.6-release-notes.md index 26fcb373689..768dae42bda 100644 --- a/doc/release-notes/5.6-release-notes.md +++ b/doc/release-notes/5.6-release-notes.md @@ -50,7 +50,7 @@ To increase support for internationalization and to improve compliance with CESS ## New JVM Options and DB Settings -:AnonymizedFieldTypeNames can be used to identify specific fields that will be anonymized when a user creates an anonymized Private URL. +:AnonymizedFieldTypeNames can be used to enable creation of anonymized Private URLs and to specify which fields will be anonymized. ## Notes for Tool Developers and Integrators From 8496b7c3db1270234ce3a5936e40af27aa0fa302 Mon Sep 17 00:00:00 2001 From: Leonid Andreev Date: Tue, 27 Jul 2021 12:32:06 -0400 Subject: [PATCH 08/11] put back the few extra lines to the upgrade instructions (about not running the commands as root, etc.; these were originally added in 5.4). #8012. --- doc/release-notes/5.6-release-notes.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/doc/release-notes/5.6-release-notes.md b/doc/release-notes/5.6-release-notes.md index 768dae42bda..c05e13e38f1 100644 --- a/doc/release-notes/5.6-release-notes.md +++ b/doc/release-notes/5.6-release-notes.md @@ -72,6 +72,14 @@ If this is a new installation, please see our [Installation Guide](https://guide 0\. These instructions assume that you've already successfully upgraded from Dataverse Software 4.x to Dataverse Software 5 following the instructions in the [Dataverse Software 5 Release Notes](https://github.com/IQSS/dataverse/releases/tag/v5.0). After upgrading from the 4.x series to 5.0, you should progress through the other 5.x releases before attempting the upgrade to 5.6. +If you are running Payara as a non-root user (and you should be!), **remember not to execute the commands below as root**. Use `sudo` to change to that user first. For example, `sudo -i -u dataverse` if `dataverse` is your dedicated application user. + +In the following commands we assume that Payara 5 is installed in `/usr/local/payara5`. If not, adjust as needed. + +`export PAYARA=/usr/local/payara5` + +(or `setenv PAYARA /usr/local/payara5` if you are using a `csh`-like shell) + 1\. Undeploy the previous version. - `$PAYARA/bin/asadmin list-applications` @@ -94,7 +102,7 @@ If this is a new installation, please see our [Installation Guide](https://guide 7\. Deploy this version. -- `$PAYARA/bin/asadmin deploy dataverse-5.6.war` +- `$PAYARA/bin/asadmin deploy dataverse-5.6.war`5.6-release-notes.md 8\. Restart payara From a8263c36d0aee38ca0298bb91b2a059a61936e53 Mon Sep 17 00:00:00 2001 From: Leonid Andreev Date: Tue, 27 Jul 2021 14:13:13 -0400 Subject: [PATCH 09/11] rearranged the payara upgrade notes a bit (#8012) --- doc/release-notes/5.6-release-notes.md | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/doc/release-notes/5.6-release-notes.md b/doc/release-notes/5.6-release-notes.md index c05e13e38f1..3928ebfa39c 100644 --- a/doc/release-notes/5.6-release-notes.md +++ b/doc/release-notes/5.6-release-notes.md @@ -40,7 +40,7 @@ A new DB Constraint has been added in this release. Full instructions on how to ### Payara 5.2021.4 (or Higher) Required -Some changes in this release require an upgrade to Payara 5.2021.4 or higher. +Some changes in this release require an upgrade to Payara 5.2021.4 or higher. (See the upgrade section). Instructions on how to update can be found in the [Payara documentation](https://docs.payara.fish/community/docs/5.2021.4/documentation/user-guides/upgrade-payara.html) We've included the necessary steps below, but we recommend that you review the Payara upgrade instructions as it could be helpful during any troubleshooting. @@ -72,6 +72,9 @@ If this is a new installation, please see our [Installation Guide](https://guide 0\. These instructions assume that you've already successfully upgraded from Dataverse Software 4.x to Dataverse Software 5 following the instructions in the [Dataverse Software 5 Release Notes](https://github.com/IQSS/dataverse/releases/tag/v5.0). After upgrading from the 4.x series to 5.0, you should progress through the other 5.x releases before attempting the upgrade to 5.6. +The steps below include a required upgrade to Payara 5.2021.4 or higher. (It is a simple matter of reusing your existing domain directory with the new distribution). But we also recommend that you review the Payara upgrade instructions as it could be helpful during any troubleshooting: [Payara documentation](https://docs.payara.fish/community/docs/5.2021.4/documentation/user-guides/upgrade-payara.html) + + If you are running Payara as a non-root user (and you should be!), **remember not to execute the commands below as root**. Use `sudo` to change to that user first. For example, `sudo -i -u dataverse` if `dataverse` is your dedicated application user. In the following commands we assume that Payara 5 is installed in `/usr/local/payara5`. If not, adjust as needed. @@ -80,19 +83,22 @@ In the following commands we assume that Payara 5 is installed in `/usr/local/pa (or `setenv PAYARA /usr/local/payara5` if you are using a `csh`-like shell) -1\. Undeploy the previous version. +1\. Undeploy the previous version - `$PAYARA/bin/asadmin list-applications` - `$PAYARA/bin/asadmin undeploy dataverse<-version>` -2\. Stop Payara and remove the generated directory + +2\. Stop Payara - `service payara stop` - `rm -rf $PAYARA/glassfish/domains/domain1/generated` 3\. Move the current Payara directory out of the way -4\. Unzip the new Payara version in its place +- `mv $PAYARA $PAYARA.MOVED` + +4\. Download the new Payara version (5.2021.4+), and unzip it in its place 5\. Replace the brand new payara/glassfish/domains/domain1 with your old, preserved `domain1` @@ -102,7 +108,7 @@ In the following commands we assume that Payara 5 is installed in `/usr/local/pa 7\. Deploy this version. -- `$PAYARA/bin/asadmin deploy dataverse-5.6.war`5.6-release-notes.md +- `$PAYARA/bin/asadmin deploy dataverse-5.6.war` 8\. Restart payara From f4c6112c48387c1e3534fab99c61e182044a0e16 Mon Sep 17 00:00:00 2001 From: Danny Brooke Date: Wed, 28 Jul 2021 10:11:38 -0400 Subject: [PATCH 10/11] updates from code review, adding in notes from recently merged PR --- doc/release-notes/5.6-release-notes.md | 8 ++++++-- doc/release-notes/6497-migrate-api.md | 7 ------- 2 files changed, 6 insertions(+), 9 deletions(-) delete mode 100644 doc/release-notes/6497-migrate-api.md diff --git a/doc/release-notes/5.6-release-notes.md b/doc/release-notes/5.6-release-notes.md index 3928ebfa39c..2801e1525b8 100644 --- a/doc/release-notes/5.6-release-notes.md +++ b/doc/release-notes/5.6-release-notes.md @@ -18,6 +18,12 @@ Dataset metadata can be retrieved, set, and updated using a new, flatter JSON-LD This development was supported by the [Research Data Alliance](https://rd-alliance.org), DANS, and Sciences PO and follows the recommendations from the [Research Data Repository Interoperability Working Group](http://dx.doi.org/10.15497/RDA00025). +### Dataset Migration API (Experimental) + +Datasets can now imported following the format of an OAI-ORE export (RDA-conformant Bags), allowing for easier migration from one Dataverse installation to another, and migration from other systems. This experimental, superuser only, endpoint also allows keeping the existing persistent identifier (where the authority and shoulder match those for which the software is configured) and publication dates. + +This development was supported by DANS and the [Research Data Alliance](https://rd-alliance.org) and follows the recommendations from the [Research Data Repository Interoperability Working Group](http://dx.doi.org/10.15497/RDA00025). + ### Direct Upload API Now Available for Adding Multiple Files Metadata to the Dataset Users can now add metadata of multiple files to the dataset once the files exists in the s3 bucket using the direct upload API. This makes direct uploads more efficient and reduces server load by only updating the dataset once instead of once per file. For more information, see the [Direct DataFile Upload/Replace API section](https://guides.dataverse.org/en/5.6/developers/s3-direct-upload-api.html) of the Dataverse Software Guides. @@ -74,7 +80,6 @@ If this is a new installation, please see our [Installation Guide](https://guide The steps below include a required upgrade to Payara 5.2021.4 or higher. (It is a simple matter of reusing your existing domain directory with the new distribution). But we also recommend that you review the Payara upgrade instructions as it could be helpful during any troubleshooting: [Payara documentation](https://docs.payara.fish/community/docs/5.2021.4/documentation/user-guides/upgrade-payara.html) - If you are running Payara as a non-root user (and you should be!), **remember not to execute the commands below as root**. Use `sudo` to change to that user first. For example, `sudo -i -u dataverse` if `dataverse` is your dedicated application user. In the following commands we assume that Payara 5 is installed in `/usr/local/payara5`. If not, adjust as needed. @@ -88,7 +93,6 @@ In the following commands we assume that Payara 5 is installed in `/usr/local/pa - `$PAYARA/bin/asadmin list-applications` - `$PAYARA/bin/asadmin undeploy dataverse<-version>` - 2\. Stop Payara - `service payara stop` diff --git a/doc/release-notes/6497-migrate-api.md b/doc/release-notes/6497-migrate-api.md deleted file mode 100644 index 7b351de0a38..00000000000 --- a/doc/release-notes/6497-migrate-api.md +++ /dev/null @@ -1,7 +0,0 @@ -# Release Highlights - -### Dataset Migration API (Experimental) - -Datasets can now imported following the format of an OAI-ORE export (RDA-conformant Bags), allowing for easier migration from one Dataverse installation to another, and migration from other systems. This experimental, super-user only, endpoint also allows keeping the existing persistent identifier (where the authority and shoulder match those for which the software is configured) and publication dates. - -This development was supported by DANS and the [Research Data Alliance](https://rd-alliance.org) and follows the recommendations from the [Research Data Repository Interoperability Working Group](http://dx.doi.org/10.15497/RDA00025). From 2f6e5c7fac3be40db003fe768fde1db2ca8e88f1 Mon Sep 17 00:00:00 2001 From: Gustavo Durand Date: Mon, 2 Aug 2021 17:08:29 -0400 Subject: [PATCH 11/11] Update 5.6-release-notes.md --- doc/release-notes/5.6-release-notes.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/release-notes/5.6-release-notes.md b/doc/release-notes/5.6-release-notes.md index 2801e1525b8..e8a82e4f3b1 100644 --- a/doc/release-notes/5.6-release-notes.md +++ b/doc/release-notes/5.6-release-notes.md @@ -78,7 +78,7 @@ If this is a new installation, please see our [Installation Guide](https://guide 0\. These instructions assume that you've already successfully upgraded from Dataverse Software 4.x to Dataverse Software 5 following the instructions in the [Dataverse Software 5 Release Notes](https://github.com/IQSS/dataverse/releases/tag/v5.0). After upgrading from the 4.x series to 5.0, you should progress through the other 5.x releases before attempting the upgrade to 5.6. -The steps below include a required upgrade to Payara 5.2021.4 or higher. (It is a simple matter of reusing your existing domain directory with the new distribution). But we also recommend that you review the Payara upgrade instructions as it could be helpful during any troubleshooting: [Payara documentation](https://docs.payara.fish/community/docs/5.2021.4/documentation/user-guides/upgrade-payara.html) +The steps below include a required upgrade to Payara 5.2021.5 or higher. (It is a simple matter of reusing your existing domain directory with the new distribution). But we also recommend that you review the Payara upgrade instructions as it could be helpful during any troubleshooting: [Payara documentation](https://docs.payara.fish/community/docs/5.2021.5/documentation/user-guides/upgrade-payara.html) If you are running Payara as a non-root user (and you should be!), **remember not to execute the commands below as root**. Use `sudo` to change to that user first. For example, `sudo -i -u dataverse` if `dataverse` is your dedicated application user. @@ -102,7 +102,7 @@ In the following commands we assume that Payara 5 is installed in `/usr/local/pa - `mv $PAYARA $PAYARA.MOVED` -4\. Download the new Payara version (5.2021.4+), and unzip it in its place +4\. Download the new Payara version (5.2021.5+), and unzip it in its place 5\. Replace the brand new payara/glassfish/domains/domain1 with your old, preserved `domain1`