Merge branch 'develop' into https-in-docs

.github/ISSUE_TEMPLATE/bug_report.md

@@ -3,7 +3,7 @@ name: Bug report
 about: Did you encounter something unexpected or incorrect in the Dataverse software?
   We'd like to hear about it!
 title: ''
-labels: ''
+labels: 'Type: Bug'
 assignees: ''
 
 ---

README.md

@@ -15,7 +15,7 @@ We love contributors! Please see our [Contributing Guide][] for ways you can hel
 
 Dataverse is a trademark of President and Fellows of Harvard College and is registered in the United States.
 
-[![Dataverse Project logo](src/main/webapp/resources/images/dataverseproject_logo.jpg?raw=true "Dataverse Project")](http://dataverse.org)
+[![Dataverse Project logo](src/main/webapp/resources/images/dataverseproject_logo.jpg "Dataverse Project")](http://dataverse.org)
 
 [![API Test Status](https://jenkins.dataverse.org/buildStatus/icon?job=IQSS-dataverse-develop&subject=API%20Test%20Status)](https://jenkins.dataverse.org/job/IQSS-dataverse-develop/)
 [![API Test Coverage](https://img.shields.io/jenkins/coverage/jacoco?jobUrl=https%3A%2F%2Fjenkins.dataverse.org%2Fjob%2FIQSS-dataverse-develop&label=API%20Test%20Coverage)](https://jenkins.dataverse.org/job/IQSS-dataverse-develop/ws/target/coverage-it/index.html)

doc/release-notes/10001-datasets-files-api-user-permissions.md

@@ -0,0 +1,13 @@
+- New query parameter `includeDeaccessioned` added to the getVersion endpoint (/api/datasets/{id}/versions/{versionId}) to consider deaccessioned versions when searching for versions.
+
+
+- New endpoint to get user permissions on a dataset (/api/datasets/{id}/userPermissions). In particular, the user permissions that this API call checks, returned as booleans, are the following:
+
+  - Can view the unpublished dataset
+  - Can edit the dataset
+  - Can publish the dataset
+  - Can manage the dataset permissions
+  - Can delete the dataset draft
+
+
+- New permission check "canManageFilePermissions" added to the existing endpoint for getting user permissions on a file (/api/access/datafile/{id}/userPermissions).

doc/release-notes/9412-markdown-previewer.md

@@ -0,0 +1 @@
+There is now a Markdown (.md) previewer: https://dataverse-guide--9986.org.readthedocs.build/en/9986/user/dataset-management.html#file-previews

doc/release-notes/9714-files-api-extension-filters.md

@@ -0,0 +1,14 @@
+The getVersionFiles endpoint (/api/datasets/{id}/versions/{versionId}/files) has been extended to support optional filtering by:
+
+- Access status: through the `accessStatus` query parameter, which supports the following values:
+
+  - Public
+  - Restricted
+  - EmbargoedThenRestricted
+  - EmbargoedThenPublic
+
+
+- Category name: through the `categoryName` query parameter. To return files to which the particular category has been added.
+
+
+- Content type: through the `contentType` query parameter. To return files matching the requested content type. For example: "image/png".

doc/release-notes/9763-versions-api-improvements.md

@@ -0,0 +1,8 @@
+# Improvements in the /versions API
+
+- optional pagination has been added to `/api/datasets/{id}/versions` that may be useful in datasets with a large number of versions;
+- a new flag `includeFiles` is added to both `/api/datasets/{id}/versions` and `/api/datasets/{id}/versions/{vid}` (true by default), providing an option to drop the file information from the output;
+- when files are requested to be included, some database lookup optimizations have been added to improve the performance on datasets with large numbers of files. 
+
+This is reflected in the [Dataset Versions API](https://guides.dataverse.org/en/9763-lookup-optimizations/api/native-api.html#dataset-versions-api) section of the Guide.
+

doc/release-notes/9785-files-api-extension-search-text.md

@@ -0,0 +1,3 @@
+The getVersionFiles endpoint (/api/datasets/{id}/versions/{versionId}/files) has been extended to support optional filtering by search text through the `searchText` query parameter.
+
+The search will be applied to the labels and descriptions of the dataset files.

doc/release-notes/9834-files-api-extension-counts.md

@@ -0,0 +1,6 @@
+Implemented the following new endpoints:
+
+- getVersionFileCounts (/api/datasets/{id}/versions/{versionId}/files/counts): Given a dataset and its version, retrieves file counts based on different criteria (Total count, per content type, per access status and per category name). 
+
+
+- setFileCategories (/api/files/{id}/metadata/categories): Updates the categories (by name) for an existing file. If the specified categories do not exist, they will be created.

doc/release-notes/9851-datafile-payload-extension-new-file-access-endpoints.md

@@ -0,0 +1,14 @@
+Implemented the following new endpoints:
+
+- userFileAccessRequested (/api/access/datafile/{id}/userFileAccessRequested): Returns true or false depending on whether or not the calling user has requested access to a particular file. 
+
+
+- hasBeenDeleted (/api/files/{id}/hasBeenDeleted): Know if a particular file that existed in a previous version of the dataset no longer exists in the latest version.
+
+
+In addition, the DataFile API payload has been extended to include the following fields:
+
+- tabularData: Boolean field to know if the DataFile is of tabular type
+
+
+- fileAccessRequest: Boolean field to know if the file access requests are enabled on the Dataset (DataFile owner) 

doc/release-notes/9852-files-api-extension-deaccession.md

@@ -0,0 +1,12 @@
+Extended the existing endpoints:
+
+- getVersionFiles (/api/datasets/{id}/versions/{versionId}/files)
+- getVersionFileCounts (/api/datasets/{id}/versions/{versionId}/files/counts)
+
+The above endpoints now accept a new boolean optional query parameter "includeDeaccessioned", which, if enabled, causes the endpoint to consider deaccessioned versions when searching for versions to obtain files or file counts.
+
+Additionally, a new endpoint has been developed to support version deaccessioning through API (Given a dataset and a version).
+
+- deaccessionDataset (/api/datasets/{id}/versions/{versionId}/deaccession)
+
+Finally, the DataFile API payload has been extended to add the field "friendlyType"

doc/release-notes/9907-files-api-counts-with-criteria.md

@@ -0,0 +1,11 @@
+Extended the getVersionFileCounts endpoint (/api/datasets/{id}/versions/{versionId}/files/counts) to support filtering by criteria.
+
+In particular, the endpoint now accepts the following optional criteria query parameters:
+
+- contentType
+- accessStatus
+- categoryName
+- tabularTagName
+- searchText
+
+This filtering criteria is the same as the one for the getVersionFiles endpoint.

doc/release-notes/9955-Signposting-updates.md

@@ -0,0 +1,7 @@
+This release fixes several issues (#9952, #9953, #9957) where the Signposting output did not match the Signposting specification. These changes introduce backward-incompatibility, but since Signposting support was added recently (in Dataverse 5.14 in PR #8981), we feel it's best to do this clean up and not support the old implementation that was not fully compliant with the spec.
+
+To fix #9952, we surround the license info with `<` and `>`.
+
+To fix #9953, we no longer wrap the response in a `{"status":"OK","data":{` JSON object. This has also been noted in the guides at https://dataverse-guide--9955.org.readthedocs.build/en/9955/api/native-api.html#retrieve-signposting-information
+
+To fix #9957, we corrected the mime/content type, changing it from `json+ld` to `ld+json`. For backward compatibility, we are still supporting the old one, for now.

doc/release-notes/9958-dataset-api-downloadsize-ignore-tabular-size.md

@@ -0,0 +1,9 @@
+Added a new optional query parameter "mode" to the "getDownloadSize" API endpoint ("api/datasets/{identifier}/versions/{versionId}/downloadsize").
+
+This parameter applies a filter criteria to the operation and supports the following values:
+
+- All (Default): Includes both archival and original sizes for tabular files
+
+- Archival: Includes only the archival size for tabular files 
+
+- Original: Includes only the original size for tabular files

doc/release-notes/9972-files-api-filter-by-tabular-tags.md

@@ -0,0 +1,3 @@
+- New query parameter `tabularTagName` added to the getVersionFiles endpoint (/api/datasets/{id}/versions/{versionId}/files) to return files to which the particular tabular tag has been added.
+
+- New endpoint to set tabular file tags via API: /api/files/{id}/metadata/tabularTags.

doc/release-notes/9995-files-api-downloadsize-with-criteria-and-deaccessioned-support.md

@@ -0,0 +1,12 @@
+Extended the getDownloadSize endpoint (/api/datasets/{id}/versions/{versionId}/downloadsize), including the following new features:
+
+- The endpoint now accepts a new boolean optional query parameter "includeDeaccessioned", which, if enabled, causes the endpoint to consider deaccessioned dataset versions when searching for versions to obtain the file total download size.
+
+
+- The endpoint now supports filtering by criteria. In particular, it accepts the following optional criteria query parameters:
+
+  - contentType
+  - accessStatus
+  - categoryName
+  - tabularTagName
+  - searchText

doc/sphinx-guides/source/_static/admin/dataverse-external-tools.tsv

@@ -2,6 +2,6 @@ Tool	Type	Scope	Description
 Data Explorer	explore	file	"A GUI which lists the variables in a tabular data file allowing searching, charting and cross tabulation analysis. See the README.md file at https://github.com/scholarsportal/dataverse-data-explorer-v2 for the instructions on adding Data Explorer to your Dataverse."
 Whole Tale	explore	dataset	"A platform for the creation of reproducible research packages that allows users to launch containerized interactive analysis environments based on popular tools such as Jupyter and RStudio. Using this integration, Dataverse users can launch Jupyter and RStudio environments to analyze published datasets. For more information, see the `Whole Tale User Guide <https://wholetale.readthedocs.io/en/stable/users_guide/integration.html>`_."
 Binder	explore	dataset	Binder allows you to spin up custom computing environments in the cloud (including Jupyter notebooks) with the files from your dataset. `Installation instructions <https://github.com/data-exp-lab/girder_ythub/issues/10>`_ are in the Data Exploration Lab girder_ythub project.
-File Previewers	explore	file	"A set of tools that display the content of files - including audio, html, `Hypothes.is <https://hypothes.is/>`_ annotations, images, PDF, text, video, tabular data, spreadsheets, GeoJSON, zip, and NcML files - allowing them to be viewed without downloading the file. The previewers can be run directly from github.io, so the only required step is using the Dataverse API to register the ones you want to use. Documentation, including how to optionally brand the previewers, and an invitation to contribute through github are in the README.md file. Initial development was led by the Qualitative Data Repository and the spreasdheet previewer was added by the Social Sciences and Humanities Open Cloud (SSHOC) project. https://github.com/gdcc/dataverse-previewers"
+File Previewers	explore	file	"A set of tools that display the content of files - including audio, html, `Hypothes.is <https://hypothes.is/>`_ annotations, images, PDF, Markdown, text, video, tabular data, spreadsheets, GeoJSON, zip, and NcML files - allowing them to be viewed without downloading the file. The previewers can be run directly from github.io, so the only required step is using the Dataverse API to register the ones you want to use. Documentation, including how to optionally brand the previewers, and an invitation to contribute through github are in the README.md file. Initial development was led by the Qualitative Data Repository and the spreasdheet previewer was added by the Social Sciences and Humanities Open Cloud (SSHOC) project. https://github.com/gdcc/dataverse-previewers"
 Data Curation Tool	configure	file	"A GUI for curating data by adding labels, groups, weights and other details to assist with informed reuse. See the README.md file at https://github.com/scholarsportal/Dataverse-Data-Curation-Tool for the installation instructions."
 Ask the Data	query	file	Ask the Data is an experimental tool that allows you ask natural language questions about the data contained in Dataverse tables (tabular data). See the README.md file at https://github.com/IQSS/askdataverse/tree/main/askthedata for the instructions on adding Ask the Data to your Dataverse installation. 

doc/sphinx-guides/source/api/dataaccess.rst

@@ -404,6 +404,18 @@ A curl example using an ``id``::
 
     curl -H "X-Dataverse-key:$API_TOKEN" -X GET http://$SERVER/api/access/datafile/{id}/listRequests
 
+User Has Requested Access to a File:
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+``/api/access/datafile/{id}/userFileAccessRequested``
+
+This method returns true or false depending on whether or not the calling user has requested access to a particular file.
+
+A curl example using an ``id``::
+
+    curl -H "X-Dataverse-key:$API_TOKEN" -X GET "http://$SERVER/api/access/datafile/{id}/userFileAccessRequested"
+
+
 Get User Permissions on a File:
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -414,6 +426,7 @@ This method returns the permissions that the calling user has on a particular fi
 In particular, the user permissions that this method checks, returned as booleans, are the following:
 
 * Can download the file
+* Can manage the file permissions
 * Can edit the file owner dataset
 
 A curl example using an ``id``::