Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data table previews #1758

Closed
laurenwalker opened this issue May 18, 2021 · 11 comments · Fixed by #2523
Closed

Data table previews #1758

laurenwalker opened this issue May 18, 2021 · 11 comments · Fixed by #2523
Assignees
Labels
ADC CI-14 Data search and display improvements (ADC deliverable) arctic data center enhancement Epic ESS-DIVE Issues associated with the ESS-DIVE project
Milestone

Comments

@laurenwalker
Copy link
Member

Display a preview of the data table so researchers can browse a few rows of data before downloading.

Milestone CI-14 in ADC proposal.

@robyngit
Copy link
Member

As part of the markdown editor, I created a TableEditorView. Just wanted to point this out in case the view can be re-purposed as a data table previewer. The view would need to be updated to include a non-editable mode, and to parse csv data (in addition to the currently supported markdown and JSON formats).

@amoeba
Copy link
Contributor

amoeba commented May 18, 2021

This would be a great addition.

Seems like we might want to tweak the behavior of MNRead.get to support either HTTP range requests and/or streaming out responses. For very small files, grabbing the entire thing in order to build a preview isn't terrible but the experience gets worse as the size of the previewed file grows. Range requests would allow us to do a lot of neat things beyond just previews too. If we did this, we might consider whether or not partial reads count as read events.

@laurenwalker
Copy link
Member Author

I think partial reads should definitely count as view events, but not as download events, which they currently would be.

@laurenwalker laurenwalker changed the title Data table previews on the dataset landing page Data table previews Dec 21, 2021
@laurenwalker
Copy link
Member Author

We also recently had a request via #1969 to display previews of .txt files as well. We should include that in our overall design for previewing data tables, too.

@robyngit robyngit added arctic data center ADC CI-14 Data search and display improvements (ADC deliverable) labels May 17, 2023
@vchendrix
Copy link
Collaborator

Hi there @robyngit I would like to +1 this issue ticket.

@robyngit robyngit added the ESS-DIVE Issues associated with the ESS-DIVE project label Oct 2, 2023
@robyngit
Copy link
Member

robyngit commented Oct 2, 2023

Thanks @vchendrix, looking into what would be involved to implement this

@robyngit
Copy link
Member

robyngit commented Oct 2, 2023

@vchendrix we discussed this as a team today. There are some backend changes required before we're able to move forward with this in MetacatUI. These are implied in the comments above, but to summarize:

  • Metacat doesn't support downloading a portion of a file, so to preview files we'd need to download the entire thing. This would be very slow, especially for large files.
  • Other than performance, the issue is that there is currently no way to download a file for preview without counting it as a download event. So a preview feature like this would skew our metrics pretty badly.

I've made in new a new Metacat issue to cover this. It's on the roadmap, but not currently a high priority for us.

@vchendrix
Copy link
Collaborator

vchendrix commented Oct 5, 2023 via email

@robyngit
Copy link
Member

robyngit commented Oct 5, 2023

Previewing only smaller and simple files was my initial thought as well @vchendrix! There would still be a problem with inflating the metrics though. We can't retrieve files without adding a "download" event to the metrics counts. So anytime someone viewed a dataset with some of the data previewed, download events would be recorded, and eventually those statistics would be inflated and not very representative anymore.

If the preview wasn't automatic, for example, a user had to click a button to explore a file, then perhaps there's an argument for counting that as a download... But probably something to get some feedback on and discuss. What do you think?

@vchendrix
Copy link
Collaborator

Previewing only smaller and simple files was my initial thought as well @vchendrix! There would still be a problem with inflating the metrics though. We can't retrieve files without adding a "download" event to the metrics counts. So anytime someone viewed a dataset with some of the data previewed, download events would be recorded, and eventually those statistics would be inflated and not very representative anymore.

If the preview wasn't automatic, for example, a user had to click a button to explore a file, then perhaps there's an argument for counting that as a download... But probably something to get some feedback on and discuss. What do you think?

That is what I had in mind. Not an automatic preview but an intentional one. And maybe "preview" is the wrong term if they are downloading and viewing the file in their browser. I was thinking that there doesn't need to be any additional javascript for viewing the file just use the browser capabilities as most browsers can render files of different types in their native format.

@robyngit
Copy link
Member

robyngit commented Jul 8, 2024

I think the MVP for this feature is:

  • Include a "preview" button for files under a maximum file size and of certain file types
  • This will open a modal with the data displayed

@robyngit robyngit self-assigned this Sep 9, 2024
@robyngit robyngit added this to the 2.32.0 milestone Sep 9, 2024
robyngit added a commit that referenced this issue Sep 10, 2024
robyngit added a commit that referenced this issue Sep 10, 2024
and remove unused imports

Issue #1758
robyngit added a commit that referenced this issue Sep 10, 2024
robyngit added a commit that referenced this issue Sep 10, 2024
robyngit added a commit that referenced this issue Sep 10, 2024
robyngit added a commit that referenced this issue Sep 10, 2024
More robust sorting for edge cases.
Fix issue caused by changing isNaN to Number.isNaN
for eslint fixes.

Issue #1758
robyngit added a commit that referenced this issue Sep 10, 2024
robyngit added a commit that referenced this issue Sep 10, 2024
robyngit added a commit that referenced this issue Sep 11, 2024
robyngit added a commit that referenced this issue Sep 11, 2024
robyngit added a commit that referenced this issue Sep 12, 2024
- Separate out the logic for getting the data from DataONE from the logic for downloading that data onto the user's machine.
- Allows us to use the same data fetching logic in other parts of the app, i.e. the DataONEObjectView.
- Also separate out the logic for getting the file name.
- Add a tests for the new functions

Issue #1758
robyngit added a commit that referenced this issue Sep 12, 2024
- MVP, shows CSVs in a table (no other file types yet, no loading spinner, or error handling)
- Ensure TableEditorView returns the view from render()

Issue #1758
robyngit added a commit that referenced this issue Sep 12, 2024
- Move rendering of the download button & view data button to a new method
- Render both buttons in a container div

Issue #1758
robyngit added a commit that referenced this issue Sep 18, 2024
- Instead of creating an empty table, then populating it, we now add table cell content as it's created
- Instead of rendering the table in editing mode then removing editing elements if the table is in view-mode, we now render the table without editing elements if the table is in view-mode
- Also use document fragment to append table rows to the table element, which is faster than appending each row individually

Issue #1758
robyngit added a commit that referenced this issue Sep 18, 2024
- Truncate data and show a message when the table is too large

Issue #1758
robyngit added a commit that referenced this issue Sep 18, 2024
- Add row numbers to parsed data rather than assuming the first column is the row number
- Handle parsing errors more gracefully
- Don't show empty CSV lines as data rows
- Allow multiple TableEditorView instances on the same page (Fix issues with dropdowns & unique IDs)
- Fix issue with adding new rows & cols in editor mode

Issue #1758
robyngit added a commit that referenced this issue Sep 18, 2024
- Show error message when file type is not supported or file size is too large

Issue #1758
robyngit added a commit that referenced this issue Sep 18, 2024
Can now initialize DataObjectView with a object ID or SolrModel

Issue #1758
robyngit added a commit that referenced this issue Sep 18, 2024
Fixes SolrResult test
Fix a typo

Issue #1758
robyngit added a commit that referenced this issue Sep 18, 2024
robyngit added a commit that referenced this issue Sep 18, 2024
@robyngit robyngit linked a pull request Sep 18, 2024 that will close this issue
4 tasks
robyngit added a commit that referenced this issue Sep 19, 2024
rushirajnenuji added a commit that referenced this issue Sep 25, 2024
MetadataView & TableEditorView cleanup (Step 1 of issue #1758)
rushirajnenuji added a commit that referenced this issue Sep 25, 2024
Support rendering TableEditorView from csv (Step 2 of issue #1758)
rushirajnenuji added a commit that referenced this issue Sep 25, 2024
Add a view-only mode to the Table Editor View (Step 3 of issue #1758)
rushirajnenuji added a commit that referenced this issue Sep 25, 2024
Create the DataObject View (Step 4 of issue #1758)
rushirajnenuji added a commit that referenced this issue Sep 26, 2024
Create the ViewObjectButtonView and use in Metadata View (Step 5 of issue #1758)
rushirajnenuji added a commit that referenced this issue Sep 26, 2024
Final misc. tasks to support data previews (Step 6 of issue #1758)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ADC CI-14 Data search and display improvements (ADC deliverable) arctic data center enhancement Epic ESS-DIVE Issues associated with the ESS-DIVE project
Development

Successfully merging a pull request may close this issue.

4 participants