Auxiliary Files - Accessing DP releases in Dataverse

[djbrooke]

Between the existing configure tools framework and the auxiliary files support we're adding in #7275/#7350, we'll soon have a way to deposit DP release files into Dataverse. Now we need to determine how to display those files to users in Dataverse. The use case around this is:

"I want to view a differentially private release of a datafile so that I can learn enough to determine whether or not it’s worth my time to go through the request process for the raw data."

We kicked this around internally and with some members of the DP team and one option may to be add some download and explore options on the Dataset and File pages, options that would be available BEFORE the user requests access to the file. Right now, the user is just presented with some file metadata and a request access option:

Instead, we could add some options for download and exploration appropriate to the auxiliary files that have been deposited, something like what we've started to investigate here: working UI & behavior document

Some considerations:

• We need to figure out the logic for auxiliary files display here. We want to keep auxiliary files general, but some DP-specific support may be required. IMHO it's OK if we build in some DP-specific stuff here and revisit when we have other use cases around aux files better defined.
• Related to the question above, we need to determine which DP files (.pdf, json, xml) are categorized under which option/dropdown - download, explore, preview.
• Is there anything we need to stop doing in Dataverse to make sure that privacy is not spoiled? As part of the ingest process we create summary stats for the raw file, so if this will reveal too much we may need to revisit to determine what to no longer make available when a DP release exists.
• If a user selects multiple files, some that have DP releases and others that don't, and clicks "Download All" - what happens? I'd be OK with handling this the same way as we currently handle multiple file download when a restricted file is selected.

[djbrooke]

Questions and discussion from design meeting 11/18:

• If you have access or are the data owner, what shows up on the access dropdown? Do you still see the aux files? One potential solution is to show all files (as the data owner may want to be able to see what people without access see)
• If there's only one option, do we provide any options in a dropdown (Ex. request access)? Is there any advantage to showing the options? There may be some benefit to showing that "request access" is the only option - otherwise in this situation someone could click "request access" and kick off a access request to a person unintentionally. Note that user confirmation is currently delivered by the "Request Access" being turned to "Access Requested" in gray - we can't do this if the option is not shown when there's only one option.
• If the only access is request access, is there some different icon that could be shown (bullhorn)? This would potentially address the concern of the user not knowing what would happen once the access button was clicked. One other here option would be to show the proper button (explore, access, request access) if only one option exists. This would revert some changes from the kebabs work.
• Is access the right symbol for all the potential options (as of 5.2 we have downloads/explore under the access dropdown)?

[TaniaSchlatter]

Slides with current status. There are a few open questions noted, to be worked out as part of development

[pdurbin]

In 7e7a502 I just pushed a branch to stub out downloading differentially private statistics. I'm having some JSF trouble (screenshot below) and the code contains several FIXMEs.

I need to go respond to feedback on a pull request so others are welcome to swarm on this.

[mheppler]

QUESTIONS: OpenDP Release Aux File #7400

system configure tools config

• QUESTION how is the aux file name "Differentially Private Statistics" added? pulled from the name of the tool? defined by us? Internationalization? (Gustavo's idea?, From Danny, "tool should be able to define the bucket/header")
• explore/any other tool changes (From Danny, "out of scope for MVP")

system storing, displaying aux files for dataset files

• aux files are for all file types (currently only tabular, see Jim's comment below)
• aux files are public for restricted files
• QUESTION include DP info in the metadata for indexing? (How might we help an end user discover DP aux files)

end user access options for tabular files (File Access > Download Options)

• QUESTION add “Differentially Private” file tag? (How might we help an end user discover DP aux files? From Danny, "tool adds the tag using existing API")
• ANSWER no changes to current Variable Metadata .XML files. Store and display the Differentially Private Statistics .XML "fuzzy stats" file version with other aux files.

end user access options for restricted files (File Access > Download Options)

• ANSWERED request access UX improvements (People requesting access to a file need visible feedback when request is sent #7469) to restricted icons (tooltips) and File Access btn render logic (always displayed for every file -- see data file citation note below) and added new File Access status to top of menu (e.g. "Restricted" plus Request Access terms tab msg "Users may not request access to files.")
• add "Users may not request access to files." msg from bundle/terms to File Access btn dropdown if request access is not enabled in dataset terms for restricted files

end user access options for ALL files (File Access > Download Options)

• data file citation download options for all file types (not just tabular, see file pg citation block download options available for all files, see mockups slide 7)

[qqmyers]

FYI: For QDR on work related to linking papers and data via Hypothesis annotations, I've been doing some incremental work to allow Aux files for other types and tracking issues (e.g. two aux files can have the same metadata). I'll get those into a PR request at some point, but if someone else is going to work on it, let me know and I'll share what I have.

[pdurbin]

I'm having some JSF trouble (screenshot below)

I just wanted to give an update that somewhat magically JSF started behaving for me as as of 3c0140b here's how the files under "Differentially Private Statistics" look:

That is, this form: lablel (tag)

I have download working but I'll note that the existing download code doesn't put any sort of extension (at least for the example in AccessIT):

Also on my mind is the isPublic boolean on the AuxiliaryFile object. If this is false, I'm not sure how the permissions are determined.

[scolapasta]

Also on my mind is the isPublic boolean on the AuxiliaryFile object. If this is false, I'm not sure how the permissions are determined.

While we may someday want something more complex, I think for this initial release is:
isPublic - public
restricted - same permissions as the main file

(I don't think we can have a case for public main file and restricted aux files)

[mheppler]

Discussed some of the questions above with Phil and implemented some of the changes to the render logic for the File Access btn as well as some of it's dropdown options. These move the UI closer to some of the changes in the mockups, but there are outstanding questions that are blockers until answered.

[pdurbin]

Here are a couple screenshots from pull request #7729 that show what was implemented. Auxiliary files uploaded as type=DP appear under "Differentially Private Statistics" under file level download. The rest appear under "Other Auxiliary Files".