Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

4406 dataverse move api #4543

Merged
merged 24 commits into from
Apr 24, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
d6620f6
basic support for moving dataverses w/ recursive indexing #4406
ferrys Feb 20, 2018
62a259b
moved recursive indexing of dataverses to IndexServiceBean #4406
ferrys Feb 20, 2018
8c5d566
removing guestbooks and templates not in new dataverse #4406
ferrys Feb 21, 2018
474049d
Removed moved dataverse as a featured dataverse #4406
ferrys Feb 26, 2018
e09ec9c
removing metadata blocks if they arent associated with destination #4406
ferrys Feb 28, 2018
426090e
Recursively removing guestbooks/templates
ferrys Mar 14, 2018
dc34c71
unit tests for movedataversecommand #4406
ferrys Mar 23, 2018
33c42d0
updated errorhandling to show all items that need to be removed #4406
ferrys Mar 26, 2018
b7c6bf4
Documentation and removing unused imports #4406
ferrys Mar 27, 2018
bf71c22
Actual curl command for admin api docs #4406
ferrys Mar 27, 2018
10fbfc4
small doc change #4406
ferrys Mar 27, 2018
09b2214
remove gigantic JSON output from list-users #4406
pdurbin Mar 27, 2018
8ffe464
Merge branch 'develop' into 4406-dataverse-move-api
ferrys Apr 2, 2018
4bc19af
Merge branch 'develop' into 4406-dataverse-move-api
ferrys Apr 3, 2018
0391888
Async indexing of dataverses #4406
ferrys Apr 9, 2018
1b44275
basic removal of linked dataverses/datasets #4406 #1364
ferrys Apr 11, 2018
108adff
Delete linking dataverses/datasets via API #4406 #1364
ferrys Apr 12, 2018
794b939
Documentation for deleting linked dataverses/datasets #4406 #1364
ferrys Apr 17, 2018
ac883a8
recursive indexing through list of ids, more logging #4406
ferrys Apr 18, 2018
c46a5ca
Fix merge conflicts with develop #4406
ferrys Apr 18, 2018
cbbff51
fixed indexing issue #4406
ferrys Apr 19, 2018
d8e376c
indexing deleted links #1364
ferrys Apr 19, 2018
4541d37
DataverseIT test for indexing in move dataverse #4406
ferrys Apr 19, 2018
9a5cc88
fixed logging and deleting links with dv move #4406
ferrys Apr 23, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions doc/sphinx-guides/source/admin/dataverses-datasets.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
Managing Datasets and Dataverses
================================

.. contents:: |toctitle|
:local:

Dataverses
----------

Move a Dataverse
^^^^^^^^^^^^^^^^

Moves a dataverse whose id is passed to a new dataverse whose id is passed. The dataverse alias also may be used instead of the id. If the moved dataverse has a guestbook, template, metadata block, link, or featured dataverse that is not compatible with the destination dataverse, you will be informed and given the option to force the move and remove the association. Only accessible to superusers. ::

curl -H "X-Dataverse-key: $API_TOKEN" -X POST http://$SERVER/api/dataverses/$id/move/$destination-id

Unlink a Dataverse
^^^^^^^^^^^^^^^^^^

Removes a link between a dataverse and another dataverse. Only accessible to superusers. ::

curl -H "X-Dataverse-key: $API_TOKEN" -X DELETE http://$SERVER/api/dataverses/$id/deleteLink/$linking-dataverse-id

Datasets
--------

Move a Dataset
^^^^^^^^^^^^^^

Moves a dataset whose id is passed to a dataverse whose alias is passed. If the moved dataset has a guestbook that is not compatible with the destination dataverse, you will be informed and given the option to force the move and remove the guestbook. Only accessible to superusers. ::

curl -H "X-Dataverse-key: $API_TOKEN" -X POST http://$SERVER/api/datasets/$id/move/$alias

Unlink a Dataset
^^^^^^^^^^^^^^^^

Removes a link between a dataset and a dataverse. Only accessible to superusers. ::

curl -H "X-Dataverse-key: $API_TOKEN" -X DELETE http://$SERVER/api/datasets/$id/deleteLink/$linking-dataverse-id
1 change: 1 addition & 0 deletions doc/sphinx-guides/source/admin/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ These "superuser" tasks are managed via the new page called the Dashboard. A use
timers
geoconnect-worldmap
user-administration
dataverses-datasets
solr-search-index
ip-groups
monitoring
Expand Down
16 changes: 16 additions & 0 deletions doc/sphinx-guides/source/admin/user-administration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,22 @@ Users are listed alphabetically by username. The search bar above the table allo

If you would like to remove all roles/permissions from a user's account (in the event of their leaving your organization, for example) then you can do so by clicking the "Remove All" button under the Roles column. This will keep the user's account active, but will revert it to put the account on the level of a default user with default permissions.

List Users via API
~~~~~~~~~~~~~~~~~~

There are two ways to list users via API. If you have relatively few users, you can get them all as a dump with this command with a superuser API token::

curl -H "X-Dataverse-key: $API_TOKEN" http://localhost:8080/api/admin/authenticatedUsers

If you have many users and want to be able to search and paginate through the results, use the command below with a superuser API token::

curl -H "X-Dataverse-key: $API_TOKEN" http://localhost:8080/api/admin/list-users

With the ``list-users`` form you can include the following optional query parameters:

* ``searchTerm`` A string that matches the beginning of a user identifier, first name, last name or email address.
* ``itemsPerPage`` The number of detailed results to return. The default is 25. This number has no limit. e.g. You could set it to 1000 to return 1,000 results
* ``selectedPage`` The page of results to return. The default is 1.

Confirm Email
-------------
Expand Down
117 changes: 0 additions & 117 deletions doc/sphinx-guides/source/api/native-api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -254,13 +254,6 @@ Now that the resulting JSON file only contains the ``metadataBlocks`` key, you c

Now that you've made edits to the metadata in your JSON file, you can send it to Dataverse as described above.

Move Dataset to Another Dataverse
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Moves a dataset whose id is passed to a dataverse whose alias is passed. Only accessible to superusers. ::

POST http://$SERVER/api/datasets/$id/move/$alias?key=$apiKey

Publish a Dataset
~~~~~~~~~~~~~~~~~

Expand Down Expand Up @@ -819,116 +812,6 @@ Creates a global role in the Dataverse installation. The data POSTed are assumed

POST http://$SERVER/api/admin/roles

List Users
~~~~~~~~~~

List users with the options to search and "page" through results. Only accessible to superusers. Optional parameters:

* ``searchTerm`` A string that matches the beginning of a user identifier, first name, last name or email address.
* ``itemsPerPage`` The number of detailed results to return. The default is 25. This number has no limit. e.g. You could set it to 1000 to return 1,000 results
* ``selectedPage`` The page of results to return. The default is 1.

::

GET http://$SERVER/api/admin/list-users


Sample output appears below.

* When multiple pages of results exist, the ``selectedPage`` parameters may be specified.
* Note, the resulting ``pagination`` section includes ``pageCount``, ``previousPageNumber``, ``nextPageNumber``, and other variables that may be used to re-create the UI.

.. code-block:: text

{
"status":"OK",
"data":{
"userCount":27,
"selectedPage":1,
"pagination":{
"isNecessary":true,
"numResults":27,
"numResultsString":"27",
"docsPerPage":25,
"selectedPageNumber":1,
"pageCount":2,
"hasPreviousPageNumber":false,
"previousPageNumber":1,
"hasNextPageNumber":true,
"nextPageNumber":2,
"startResultNumber":1,
"endResultNumber":25,
"startResultNumberString":"1",
"endResultNumberString":"25",
"remainingResults":2,
"numberNextResults":2,
"pageNumberList":[
1,
2
]
},
"bundleStrings":{
"userId":"ID",
"userIdentifier":"Username",
"lastName":"Last Name ",
"firstName":"First Name ",
"email":"Email",
"affiliation":"Affiliation",
"position":"Position",
"isSuperuser":"Superuser",
"authenticationProvider":"Authentication",
"roles":"Roles",
"createdTime":"Created Time",
"lastLoginTime":"Last Login Time",
"lastApiUseTime":"Last API Use Time"
},
"users":[
{
"id":8,
"userIdentifier":"created1",
"lastName":"created1",
"firstName":"created1",
"email":"created1@g.com",
"affiliation":"hello",
"isSuperuser":false,
"authenticationProvider":"BuiltinAuthenticationProvider",
"roles":"Curator",
"createdTime":"2017-06-28 10:36:29.444"
},
{
"id":9,
"userIdentifier":"created8",
"lastName":"created8",
"firstName":"created8",
"email":"created8@g.com",
"isSuperuser":false,
"authenticationProvider":"BuiltinAuthenticationProvider",
"roles":"Curator",
"createdTime":"2000-01-01 00:00:00.0"
},
{
"id":1,
"userIdentifier":"dataverseAdmin",
"lastName":"Admin",
"firstName":"Dataverse",
"email":"dataverse@mailinator2.com",
"affiliation":"Dataverse.org",
"position":"Admin",
"isSuperuser":true,
"authenticationProvider":"BuiltinAuthenticationProvider",
"roles":"Admin, Contributor",
"createdTime":"2000-01-01 00:00:00.0",
"lastLoginTime":"2017-07-03 12:22:35.926",
"lastApiUseTime":"2017-07-03 12:55:57.186"
}

// ... 22 more user documents ...
]
}
}

.. note:: "List all users" ``GET http://$SERVER/api/admin/authenticatedUsers`` is deprecated, but supported.

List Single User
~~~~~~~~~~~~~~~~

Expand Down
4 changes: 4 additions & 0 deletions src/main/java/Bundle.properties
Original file line number Diff line number Diff line change
Expand Up @@ -1668,6 +1668,10 @@ find.dataset.error.dataset_id_is_null=When accessing a dataset based on Persiste
find.dataset.error.dataset.not.found.persistentId=Dataset with Persistent ID {0} not found.
find.dataset.error.dataset.not.found.id=Dataset with ID {0} not found.
find.dataset.error.dataset.not.found.bad.id=Bad dataset ID number: {0}.
find.datasetlinking.error.not.found.ids=Dataset linking dataverse with dataset ID {0} and dataset linking dataverse ID {1} not found.
find.datasetlinking.error.not.found.bad.ids=Bad dataset ID number: {0} or dataset linking dataverse ID number: {1}.
find.dataverselinking.error.not.found.ids=Dataverse linking dataverse with dataverse ID {0} and dataverse linking dataverse ID {1} not found.
find.dataverselinking.error.not.found.bad.ids=Bad dataverse ID number: {0} or dataverse linking dataverse ID number: {1}.
file.addreplace.error.dataset_id_not_found=There was no dataset found for ID:
file.addreplace.error.no_edit_dataset_permission=You do not have permission to edit this dataset.
file.addreplace.error.filename_undetermined=The file name cannot be determined.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@

import java.util.ArrayList;
import java.util.List;
import java.util.logging.Logger;
import javax.ejb.Stateless;
import javax.inject.Named;
import javax.persistence.EntityManager;
Expand All @@ -21,6 +22,7 @@
@Stateless
@Named
public class DatasetLinkingServiceBean implements java.io.Serializable {
private static final Logger logger = Logger.getLogger(DatasetLinkingServiceBean.class.getCanonicalName());

@PersistenceContext(unitName = "VDCNet-ejbPU")
private EntityManager em;
Expand Down Expand Up @@ -63,6 +65,19 @@ public List<Dataverse> findLinkingDataverses(Long datasetId) {
}
return retList;
}

public DatasetLinkingDataverse findDatasetLinkingDataverse(Long datasetId, Long linkedDataverseId) {
DatasetLinkingDataverse foundDatasetLinkingDataverse = null;
try {
foundDatasetLinkingDataverse = em.createQuery("SELECT OBJECT(o) FROM DatasetLinkingDataverse AS o WHERE o.linkingDataverse.id = :dataverseId AND o.dataset.id = :datasetId", DatasetLinkingDataverse.class)
.setParameter("datasetId", datasetId)
.setParameter("dataverseId", linkedDataverseId)
.getSingleResult();
} catch (javax.persistence.NoResultException e) {
logger.fine("no datasetLinkingDataverse found for datasetId " + datasetId + " and linkedDataverseId " + linkedDataverseId);
}
return foundDatasetLinkingDataverse;
}

public List<DatasetLinkingDataverse> findDatasetLinkingDataverses(Long datasetId) {
return em.createQuery("select object(o) from DatasetLinkingDataverse as o where o.dataset.id =:datasetId order by o.id", DatasetLinkingDataverse.class)
Expand All @@ -77,7 +92,7 @@ public void save(DatasetLinkingDataverse datasetLinkingDataverse) {
em.merge(datasetLinkingDataverse);
}
}

public boolean alreadyLinked(Dataverse dataverse, Dataset dataset) {
TypedQuery<DatasetLinkingDataverse> typedQuery = em.createQuery("SELECT OBJECT(o) FROM DatasetLinkingDataverse AS o WHERE o.linkingDataverse.id = :dataverseId AND o.dataset.id = :datasetId", DatasetLinkingDataverse.class);
typedQuery.setParameter("dataverseId", dataverse.getId());
Expand Down
22 changes: 22 additions & 0 deletions src/main/java/edu/harvard/iq/dataverse/DatasetServiceBean.java
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,28 @@ private List<Dataset> findByOwnerId(Long ownerId, boolean onlyPublished) {
return retList;
}
}

public List<Long> findIdsByOwnerId(Long ownerId) {
return findIdsByOwnerId(ownerId, false);
}

private List<Long> findIdsByOwnerId(Long ownerId, boolean onlyPublished) {
List<Long> retList = new ArrayList<>();
if (!onlyPublished) {
TypedQuery<Long> query = em.createQuery("select o.id from Dataset as o where o.owner.id =:ownerId order by o.id", Long.class);
query.setParameter("ownerId", ownerId);
return query.getResultList();
} else {
TypedQuery<Dataset> query = em.createQuery("select object(o) from Dataset as o where o.owner.id =:ownerId order by o.id", Dataset.class);
query.setParameter("ownerId", ownerId);
for (Dataset ds : query.getResultList()) {
if (ds.isReleased() && !ds.isDeaccessioned()) {
retList.add(ds.getId());
}
}
return retList;
}
}

public List<Dataset> findAll() {
return em.createQuery("select object(o) from Dataset as o order by o.id", Dataset.class).getResultList();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@

import java.util.ArrayList;
import java.util.List;
import java.util.logging.Logger;
import javax.ejb.EJB;
import javax.ejb.Stateless;
import javax.inject.Named;
Expand All @@ -22,6 +23,7 @@
@Stateless
@Named
public class DataverseLinkingServiceBean implements java.io.Serializable {
private static final Logger logger = Logger.getLogger(DataverseLinkingServiceBean.class.getCanonicalName());

@PersistenceContext(unitName = "VDCNet-ejbPU")
private EntityManager em;
Expand Down Expand Up @@ -59,6 +61,19 @@ public void save(DataverseLinkingDataverse dataverseLinkingDataverse) {
em.merge(dataverseLinkingDataverse);
}
}

public DataverseLinkingDataverse findDataverseLinkingDataverse(Long dataverseId, Long linkedDataverseId) {
DataverseLinkingDataverse foundDataverseLinkingDataverse = null;
try {
foundDataverseLinkingDataverse = em.createQuery("SELECT OBJECT(o) FROM DataverseLinkingDataverse AS o WHERE o.linkingDataverse.id = :linkedDataverseId AND o.dataverse.id = :dataverseId", DataverseLinkingDataverse.class)
.setParameter("dataverseId", dataverseId)
.setParameter("linkedDataverseId", linkedDataverseId)
.getSingleResult();
} catch (javax.persistence.NoResultException e) {
logger.fine("No DataverseLinkingDataverse found for dataverseId " + dataverseId + " and linkedDataverseId " + linkedDataverseId);
}
return foundDataverseLinkingDataverse;
}

public boolean alreadyLinked(Dataverse definitionPoint, Dataverse dataverseToLinkTo) {
TypedQuery<DataverseLinkingDataverse> typedQuery = em.createQuery("SELECT OBJECT(o) FROM DataverseLinkingDataverse AS o WHERE o.linkingDataverse.id = :dataverseId AND o.dataverse.id = :dataverseToLinkTo", DataverseLinkingDataverse.class);
Expand Down
41 changes: 41 additions & 0 deletions src/main/java/edu/harvard/iq/dataverse/DataverseServiceBean.java
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,11 @@ public List<Dataverse> findByOwnerId(Long ownerId) {
return em.createQuery(qr, Dataverse.class).setParameter("ownerId", ownerId).getResultList();
}

public List<Long> findIdsByOwnerId(Long ownerId) {
String qr = "select o.id from Dataverse as o where o.owner.id =:ownerId order by o.id";
return em.createQuery(qr, Long.class).setParameter("ownerId", ownerId).getResultList();
}

public List<Dataverse> findPublishedByOwnerId(Long ownerId) {
String qr ="select object(o) from Dataverse as o where o.owner.id =:ownerId and o.publicationDate is not null order by o.name";
return em.createQuery(qr, Dataverse.class).setParameter("ownerId", ownerId).getResultList();
Expand Down Expand Up @@ -558,4 +563,40 @@ public void populateDvSearchCard(SolrSearchResult solrSearchResult) {
}
}
}

// function to recursively find ids of all children of a dataverse that
// are also of type dataverse
public List<Long> findAllDataverseDataverseChildren(Long dvId) {
// get list of Dataverse children
List<Long> dataverseChildren = findIdsByOwnerId(dvId);

if (dataverseChildren == null) {
return dataverseChildren;
} else {
List<Long> newChildren = new ArrayList<>();
for (Long childDvId : dataverseChildren) {
newChildren.addAll(findAllDataverseDataverseChildren(childDvId));
}
dataverseChildren.addAll(newChildren);
return dataverseChildren;
}
}

// function to recursively find ids of all children of a dataverse that are
// of type dataset
public List<Long> findAllDataverseDatasetChildren(Long dvId) {
// get list of Dataverse children
List<Long> dataverseChildren = findIdsByOwnerId(dvId);
// get list of Dataset children
List<Long> datasetChildren = datasetService.findIdsByOwnerId(dvId);

if (dataverseChildren == null) {
return datasetChildren;
} else {
for (Long childDvId : dataverseChildren) {
datasetChildren.addAll(findAllDataverseDatasetChildren(childDvId));
}
return datasetChildren;
}
}
}
Loading