Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dataverse Store #5688

Closed
4tikhonov opened this issue Mar 27, 2019 · 11 comments
Closed

Dataverse Store #5688

4tikhonov opened this issue Mar 27, 2019 · 11 comments
Assignees
Labels
Type: Suggestion an idea User Role: Superuser Has access to the superuser dashboard and cares about how the system is configured

Comments

@4tikhonov
Copy link
Contributor

4tikhonov commented Mar 27, 2019

Hi all,

In the new SSHOC DataverseEU project we're thinking about building application layer around Dataverse as we need viewers and external tools to be plugged in easily, at different levels of maturity. The biggest problem is efficient apps maintenance as some tools can die and go out of the support, or have security holes or bugs.

I have a vision that it can be kind of Dataverse store available in the dashboard where users can select software they want to install or remove from the running system.

Indeed it's going to be a mess if everything will be integrated in the core system, it can break Dataverse. I have an idea to build kind of universal Docker image for web applications (Apache/Ngix with CORS available) that will download source code of selected applications from different GitHub repositories, build, pass all tests automatically with Selenium and will be finally available for users. For Javascript frameworks it's feasible to do right now.

Dataverse already has very rich set of APIs so I think with Docker integration we can make tools and VMs secure enough to operate in the common workspace. I'm only deeply concerned about exposing sensitive data so we can even offer encryption tools to encode/decode restricted content coming to browser.

Please let me know if you think it can be something useful for you as well.

@RightInTwo
Copy link
Contributor

RightInTwo commented Mar 28, 2019

Hi Slava, I think this would be very useful!

a) I see it as a prerequisite that Dataverse + Docker is a stable and supported setup for building something as close to the core project as a "Dataverse Store". Is that the right way of thinking?

b) What kind of software would be relevant for this? Do you think it makes sense to feature a list of candidates here? Like...

  • TwoRavens
  • Whole Tale
  • DMPRoadmap
  • a connector to a custom CRIS
  • a workflow management software
  • ...

@pdurbin
Copy link
Member

pdurbin commented Apr 11, 2019

I just spotted a slide about a (future) Dataverse App Store website over at https://twitter.com/4tykhonov/status/1116229640232873984

dataversesshocdevelopment-190411063200-02

@4tikhonov I really appreciate you laying out your vision for more modularity in Dataverse! Great stuff! 🎉 Please let us know how we can help!

@RightInTwo one kind of software we talked about in our design meeting yesterday was the idea of "preview" tools at the file level (#3758). For example, what if instead of a large icon for a tabular file (screenshot below) a preview appeared? What would the look like? Would it be interactive? #3758 is the place for a deeper dive on preview at the file level but this is just an example of what a future app store for Dataverse could have in it. Please keep the ideas coming! 💡 💡 💡

Screen Shot 2019-04-11 at 8 01 16 AM

@RightInTwo
Copy link
Contributor

@4tikhonov wrote:

I have an idea to build kind of universal Docker image for web applications (Apache/Ngix with CORS available) that will download source code of selected applications from different GitHub repositories, build, pass all tests automatically

Since there are plenty of free and open container managers with GUIs available, why not choose one of them and provide templates (incl. automatic integration with each other by setting the correct ports or exchanging secrets) to set up and maintain an environment of research data software around Dataverse?

@RightInTwo
Copy link
Contributor

The idea "Monitoring dashboard for shared resources" #5760 would fit very well with this.

@pdurbin
Copy link
Member

pdurbin commented Jul 17, 2019

I'm cross-posting a new spreadsheet I just mentioned at #5028 (comment)

It's called "External Tools" and here's a link: https://docs.google.com/spreadsheets/d/1OwIxpgpWVPDPSFwDsnPfk8ivNRXUIiaAlnbFccCnfsQ/edit?usp=sharing

Here's a screenshot:

61395469-8108b080-a893-11e9-830d-b8e5e900a7c1

This is my attempt to understand what external tools exist or are being talked about. It feels related enough to this concept of an "app store" for Dataverse to mention it here in this issue.

Comments are open on that spreadsheet. Feedback is welcome here or there or anywhere. 😄

@pdurbin
Copy link
Member

pdurbin commented Oct 9, 2019

A few things.

In addition to that Google spreadsheet above where I tried to start classifying various external tools, I also refactored the guides themselves so that external tools are now in a TSV file: http://guides.dataverse.org/en/4.17/_downloads/dataverse-external-tools.tsv

I did this so that the tools could appear on multiple pages in the guides but a side effect of this is that @4tikhonov or others could treat this TSV as data. The columns are:

  • name
  • type (explore or configure)
  • scope (dataset or file)
  • description

Here's how I explain the new "dataverse-external-tools.tsv" file at http://guides.dataverse.org/en/4.17/api/external-tools.html#adding-your-tool-to-the-inventory-of-external-tools

Screen Shot 2019-10-09 at 6 58 02 AM

The other thing is that I keep dreaming that https://demo.dataverse.org (or some other server) could be powered by Kubernetes and all these tools could be showcased together. I wrote about this in a couple places:

Finally, over at #6256 (comment) I saw @4tikhonov just posted the slide about a Dataverse App Store. Great stuff!

66468576-aff38480-ea86-11e9-87d4-d7f9674527af

@4tikhonov
Copy link
Contributor Author

Hi @pdurbin, I have the same dream. If you will follow this
instruction then you'll get Dataverse with all integrated tools in Kubernetes, we did the integration of external tools in the separate Docker container.

And more interesting integrations are coming from us, just wait for the official announcement.

@pdurbin
Copy link
Member

pdurbin commented Oct 10, 2019

@4tikhonov great!

What's the "definition of done" for this issue? When do we say, "Now there is a Dataverse App Store"?

I should just wait for the announcement? 😄

@pdurbin pdurbin added Type: Suggestion an idea Status: Still Interested? User Role: Superuser Has access to the superuser dashboard and cares about how the system is configured labels Oct 10, 2022
@pdurbin
Copy link
Member

pdurbin commented Oct 10, 2022

@4tikhonov are you still interested in this? If so, what's the next step? Thanks.

@pdurbin
Copy link
Member

pdurbin commented Mar 14, 2023

@pdurbin pdurbin closed this as completed Oct 7, 2023
@pdurbin
Copy link
Member

pdurbin commented Jul 8, 2024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: Suggestion an idea User Role: Superuser Has access to the superuser dashboard and cares about how the system is configured
Projects
None yet
Development

No branches or pull requests

4 participants