Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add "Published" feature to dashboards #4725

Merged
merged 51 commits into from
Jul 11, 2019
Merged

Add "Published" feature to dashboards #4725

merged 51 commits into from
Jul 11, 2019

Conversation

Tresdon
Copy link
Contributor

@Tresdon Tresdon commented Mar 30, 2018

This feature allows users to “publish” dashboards which essentially adds a notion of dashboards being public or private. The goal of this is to reduce the clutter that comes from a whole organization creating dashboards, many of which are not relevant to other users (or only a subset, whom they can share with selectively). The changes affect the following contexts:

On creating a dashboard:
- Add a boolean published column to the dashboard model and allow the owners of a given dashboard to toggle this field in the UI.

On listing dashboards:
- Don’t show all dashboards but instead only list dashboards with the following properties
1. Dashboards the current user owns
2. Dashboards that have been published (by any user)
3. Dashboards which the user has favorited

Regarding point (3) above – The idea here was that no dashboard is strictly “private” in that it cannot be viewed if it is not published. This means that if a user creates an unpublished dashboard, they should still be able to send the link to others to share it with them. However, since the dashboard is unpublished the visitor wouldn’t see it in the list so they would need to keep the link around which isn’t a very friendly user experience. So to get around this we allow the user to favorite the dashboard and then it will show up in their list of dashboards.

issues which might be relevant to this: #1799 #2792 #4002

Tasks:

  • Add the 'published' column to the dashboard model and generate alembic migration file
  • Don't show unpublished dashboards on listing all dashboards.
  • Expose endpoint for toggling published status.
  • Protect published endpoint by allowing only owners to change the status.
  • Create React component on dashboards to represent and toggle status.
  • Show error in the UI if somebody tries to toggle published on a dashboard which they don't own
  • Show Users favorited dashboards in the dashboard list even if they're not published
  • Write unit tests for dashboard visibility
  • Show interactions with the feature in this ticket

@mistercrunch
Copy link
Member

This is neat. A few thoughts:

  • You probably didn't mean to publish superset/assets/yarn.lock as part of this PR
  • My main concern is around launching this feature, all dashboards will suddenly become invisible, I'm not sure how to mitigate this, maybe a database script that looks for all active dashboards and set them as published? An active dashboard could be defined as one that a non-owner has accessed over the past 28 days or something?
  • There needs to be an easy way for the owner to see the state publish / not published and to toggle it from the dashboard view itself, not from the CRUD view. We're trying to move away from our reliance on CRUD
  • Similar feature could be even more relevant for Charts, where the clutter is much higher, maybe apply the same consistent pattern there?
  • Curious to see what kind of SQL FAB generates when applying both base_filters, any way you can find out and share that? Let's make sure it's not doing anything crazy like running two SQL queries and showing the UNION. We should make sure this runs fast in envrionments where there are thousands of dashboards like at Airbnb.

@Tresdon
Copy link
Contributor Author

Tresdon commented Apr 6, 2018

Thanks for such useful and quick feedback on this, A couple thoughts and things to do yet

  • Ah yeah the yarn.lock did creep in there. This is probably a silly question but what would the best way be to go about getting the proper version of this? Probably just using the one from the commit before or should redoing the yarn install set it straight?
  • Releasing the feature: Perhaps it'd be fine to just have a script that sets all dashboards to published since this wouldn't affect the system at all and then people would just be able to modify it from then on? Even though 28 days-ish seems like a reasonable cutoff, this just seems minimally invasive.
  • Toggling status in dashboard view and not just CRUD: Great idea, this commit adds some of that functionality. I've added an icon right next to the favorite icon to show whether or not it's published. I built it primarily off of the FavStar component as well. It's not currently working completely.
  • About chart privacy: The charts do get more cluttered but I don't know if they're browsed as a final product in the same way as dashboards. Also wondering how this would interact with adding charts to dashboards, would people only be able to see published charts?
  • SQL Generated: I'd be interested in knowing this too. I know that I can go to each of the Filters and print out the query before and after any transformations but am not sure how to find out what things look like before applying either filter and afterwards. Any quick ideas on how to do this? I haven't looked into it much.

Things to do:

  • The route I created for modifying the status superset/dashboard/<id>/published/<action> returns a 302. I haven't dug into it too much yet but perhaps you know what I need to do to address this.
  • Show user their favorited dashboards in the list.
  • Test non-owners trying to modify the published status
  • Show generated SQL from using multiple base_filters

Thanks again for your feedback!

@codecov-io
Copy link

codecov-io commented Apr 9, 2018

Codecov Report

Merging #4725 into master will decrease coverage by 0.01%.
The diff coverage is 60.56%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master   #4725      +/-   ##
=========================================
- Coverage   63.51%   63.5%   -0.02%     
=========================================
  Files         360     361       +1     
  Lines       22890   22955      +65     
  Branches     2549    2551       +2     
=========================================
+ Hits        14539   14577      +38     
- Misses       8336    8363      +27     
  Partials       15      15
Impacted Files Coverage Δ
...ssets/src/dashboard/containers/DashboardHeader.jsx 100% <ø> (ø) ⬆️
...et/assets/src/dashboard/reducers/dashboardState.js 92.1% <0%> (-2.49%) ⬇️
superset/models/core.py 85.12% <100%> (+0.02%) ⬆️
...uperset/assets/src/dashboard/components/Header.jsx 58.94% <100%> (+0.43%) ⬆️
superset/views/core.py 73.46% <50%> (-0.54%) ⬇️
...set/assets/src/dashboard/actions/dashboardState.js 45.39% <52.38%> (+1.11%) ⬆️
...ssets/src/dashboard/components/PublishedStatus.jsx 88.88% <88.88%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5966a67...ffceb76. Read the comment docs.

@daniel5001
Copy link

Hey guys,

Any update on this merge? Would love to be able to roll this feature out.

Thanks.

@SpyderRivera
Copy link
Contributor

I completely agree "Releasing the feature: Perhaps it'd be fine to just have a script that sets all dashboards to published since this wouldn't affect the system at all and then people would just be able to modify it from then on? "

@Tresdon
Copy link
Contributor Author

Tresdon commented Apr 18, 2018

Hey @daniel5001 ,

This feature is still in the works and hasn't been forgotten, I'm currently working on ironing out a few bugs I've found as a result of adding an icon to toggle the status in the UI (and just trying to make it pleasant since it would show up on every dashboard) but other than that it's becoming pretty close to being finished!

@Tresdon
Copy link
Contributor Author

Tresdon commented May 7, 2018

a quick visual tour of the things that this PR changes:

Initial list of dashboards

image

After John publishes his secret dashboard

image

Toggling the published status of a dashboard you own

image

Trying to toggle the status of a dashboard you do not own

image

@Tresdon Tresdon changed the title [WIP] Add "Published" feature to dashboards Add "Published" feature to dashboards May 9, 2018
@Tresdon
Copy link
Contributor Author

Tresdon commented May 9, 2018

@mistercrunch This pull request should be finished now, let me know what you think. One thing I didn't mention was that this doesn't get rid of the self.has_all_datasource_access() check on dashboards and it still validates that user has access to all charts in the particular dashboard if they don't have access to all datasources. So right now for gamma users this is an additional constraint on top of them having access to all charts which we may want to change.

@SpyderRivera
Copy link
Contributor

SpyderRivera commented May 10, 2018

LGTM, I checked functionality not code

@daniel5001
Copy link

Looks awesome @Tresdon can't thank you enough for putting this together. It's a real lifesaver when utilizing this tool in a large enterprise.

@daniel5001
Copy link

@mistercrunch anything left to be done here? Looking forward to this feature. Thanks!

@SpyderRivera
Copy link
Contributor

bitmoji

@Tresdon
Copy link
Contributor Author

Tresdon commented Aug 1, 2018

@mistercrunch I've updated this feature to work with the dashboard improvements, anything else you'd like to see here?

@SpyderRivera
Copy link
Contributor

@Tresdon can you fix the dashboard tests issue?

@sylvia-tomiyama
Copy link

Some thoughts on the product experience:
What does publish vs unpublish mean? It looks like there might be some misunderstanding on this thread about what published actually means, and it would be good to clarify before rollout. Is an unpublished dashboard NOT viewable, or is it viewable but less discoverable? The original comment by @Tresdon explicitly says that it IS viewable though less discoverable ("The idea here was that no dashboard is strictly “private” in that it cannot be viewed if it is not published."), but @mistercrunch 's comment about migration and @Tresdon's reply about it seems to imply that unpublished dashboards are NOT viewable. I could see this being a successful feature in either of these cases, but I think we should be super clear about which one it is and make sure the product experience matches up with these expectations. I also think it's worth considering whether the intention is to have this status mean public vs private; or prod vs in-progress?

My understanding is that unpublished dashboards are viewable, but less discoverable. Specifically, the only differences between published vs unpublished dashboard are (a) When viewing the dashboard, the eye icon has a strikethrough or not and (b) whether it shows up in the list of dashboards (on the /welcome page, not the crud UI). My feedback is based on this understanding.

  • We may want to consider a more visible/text-based design for publish vs unpublished in the context of viewing a dashboard. For one, it seems potentially misleading to have an eye icon that's crossed out if that dashboard isn't actually hidden (since it's viewable, even if it's unpublished, and a crossed-out eye seems to imply you can't see it). Second, if we are going to make the dashboard viewable even when it's unpublished, then I think a lot of the value of a "published" status is in communicating the intent of the dashboard creator -- whether it's ready for general consumption or it's a work-in-progress/private audience. I think the icon is a little too subtle right now for communicating this important piece of information.
  • The interaction for making a dashboard "published" might belong better in the action menu for dashboards, since a natural workflow might be create->edit->publish dashboards. It's also an action that only the dashboard owner can take, so naturally it fits better with the other dashboard owner actions, vs the favoriting action which is an action that anyone can take across any dashboard.
    image
  • I'm not sure how much the /welcome page is used for people to discover dashboards, and therefore I'm not sure how much of an impact there will be from hiding unpublished dashboards from that page.
  • It might be helpful for the CRUD view to show whether a dashboard is published or not, and to be able to filter on status (not sure if this is already the plan).

@Tresdon
Copy link
Contributor Author

Tresdon commented Aug 16, 2018

Hey there @sylvia-tomiyama,

Thanks for taking a look at this and giving the user's experience some thought. I agree that some of the terminology surrounding the feature needs to be defined more clearly since there's some ambiguity around the term "published". You hit the nail on the head when you said that My understanding is that unpublished dashboards are viewable, but less discoverable – which is further bolstered by your distinction of prod vs in-progress. I think the comment about migration (which mentioned that dashboards would all of the sudden be gone) was referring to the visibility of them within these list views but they would not actually be gone or become truly unviewable. With that in mind I've considered some of the excellent points you've made and here's what I think:

Welcome List & CRUD View List for Dashboards

You mentioned as a part of your understanding that the visibility of the dashboard will change on the /welcome page but not the crud view. This wasn't quite my intention since when I was developing the feature these both used the same /dashboardmodelview/list endpoint and displayed the same dashboards. However I think it's a good idea to show admins all of the dashboards on the crud list view and to include a column of whether or not it's published for filtering. The alternative would make it so that even admins wouldn't be able to see these dashboards unless they performed a query which is less than ideal.

Right now I say that only admins should be able to see all dashboards because I believe this view is used more frequently for finding dashboards than /welcome (considering I'm not aware of a way to return to /welcome except by setting the URL specifically). If the Superset icon in the top left returned the user to this view instead of the profile I would say it should affect just the welcome page and not the CRUD view but since most people use the CRUD view to find dashboards once they're in the application I think the feature should de-clutter that view as well. Perhaps the profile button under the human icon in the top-right can become the /profile page and then an edit button on that page can take you to /users/userinfo? It seems like this may be more intuitive even without this feature we're discussing haha.

Displaying and Altering the Published Status

I think you make a great point about the eye icon being a bit misleading and a bit too subtle. I think a better solution might be something like a [draft] badge next to the title of the dashboard (possibly with a hover-over tooltip) which indicates a few things:

  • The dashboard they're viewing will not show up in the list of dashboards, they should favorite it to see it in that list along with the others
  • The elements used in the dashboard are tentative and subject to change
  • Users should consider whether or not it is okay for them to share the dashboard before doing so

This indicator seems like it would make the intention of the feature much more understandable.

As for altering the status, I think you're totally right that it makes more sense to put this in the actions dropdown especially if we forgo the icon which was both the indicator and the toggle.

The Way Forward

Here's a couple of action items which I think are appropriate after this discussion for me to work on:

  • Replace the eye icon with a [draft] badge of sorts next to the title of unpublished dashboards.
  • Add ability to toggle published status in the action dropdown menu
  • Add published field to CRUD listing of dashboards for sorting / filtering

Let me know what you think and thanks again for your consideration :)

michellethomas and others added 5 commits September 18, 2018 15:30
- The eye next to the title has been replaced with a [draft] badge
- Published status is now toggled in the Header Action Dropdown
- CRUD list shows published status
@codecov-io
Copy link

Codecov Report

Merging #4725 into master will decrease coverage by <.01%.
The diff coverage is 64.1%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #4725      +/-   ##
==========================================
- Coverage   63.72%   63.71%   -0.01%     
==========================================
  Files         386      387       +1     
  Lines       23532    23601      +69     
  Branches     2621     2627       +6     
==========================================
+ Hits        14996    15038      +42     
- Misses       8523     8550      +27     
  Partials       13       13
Impacted Files Coverage Δ
...ssets/src/dashboard/containers/DashboardHeader.jsx 100% <ø> (ø) ⬆️
...et/assets/src/dashboard/reducers/dashboardState.js 91.78% <0%> (-2.59%) ⬇️
...uperset/assets/src/dashboard/components/Header.jsx 58.94% <100%> (+0.43%) ⬆️
superset/models/core.py 85.12% <100%> (+0.02%) ⬆️
...set/assets/src/dashboard/actions/dashboardState.js 45.39% <52.38%> (+1.11%) ⬆️
superset/views/core.py 73.68% <54.83%> (-0.54%) ⬇️
...src/dashboard/components/HeaderActionsDropdown.jsx 81.48% <83.33%> (+0.23%) ⬆️
...ssets/src/dashboard/components/PublishedStatus.jsx 93.75% <93.75%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 24be692...746eb30. Read the comment docs.

@codecov-io
Copy link

codecov-io commented Sep 19, 2018

Codecov Report

Merging #4725 into master will increase coverage by 0.06%.
The diff coverage is 43.47%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #4725      +/-   ##
==========================================
+ Coverage   65.73%   65.79%   +0.06%     
==========================================
  Files         459      460       +1     
  Lines       21986    22046      +60     
  Branches     2415     2419       +4     
==========================================
+ Hits        14452    14506      +54     
- Misses       7413     7418       +5     
- Partials      121      122       +1
Impacted Files Coverage Δ
...uperset/assets/src/dashboard/components/Header.jsx 42.2% <ø> (ø) ⬆️
...ssets/src/dashboard/containers/DashboardHeader.jsx 100% <ø> (ø) ⬆️
superset/assets/src/dashboard/util/propShapes.jsx 100% <ø> (ø) ⬆️
...t/assets/src/dashboard/reducers/getInitialState.js 0% <ø> (ø) ⬆️
...et/assets/src/dashboard/reducers/dashboardState.js 83.33% <0%> (-2.2%) ⬇️
superset/models/core.py 83.66% <100%> (+0.64%) ⬆️
superset/views/core.py 72.48% <50%> (-0.39%) ⬇️
...ssets/src/dashboard/components/PublishedStatus.jsx 54.54% <54.54%> (ø)
...set/assets/src/dashboard/actions/dashboardState.js 36.55% <8.33%> (-2.55%) ⬇️
... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 61281d1...37f140a. Read the comment docs.

Tresdon Jones added 4 commits September 20, 2018 15:13
Add some tests to make sure the published status is rendered and
Make it so that users cannot see dashboards that are published
if they don't have access to any of the slices within
Some tests are failing in travis CI and not locally so
I would like to see why
@mistercrunch mistercrunch added the .pinned Draws attention label Jun 4, 2019
@1AB9502
Copy link

1AB9502 commented Jun 6, 2019

@mistercrunch ,

I would like to take over getting this story merged since @Tresdon got pulled into other things.
Would it be easier to open a new pull request, or figure out a way to resolve the merge conflicts in this pull request?

@mistercrunch
Copy link
Member

If you have write access to @Tresdon 's fork you could take over it. Otherwise you'll have to open a new PR off of your fork...

@Tresdon
Copy link
Contributor Author

Tresdon commented Jun 11, 2019

Thanks for the assistance @1AB9502!

@Tresdon
Copy link
Contributor Author

Tresdon commented Jun 24, 2019

@mistercrunch @hughhhh

I found some time to get this green again – I've also set up @1AB9502 with access to my fork so we can tag team the issue in the future.

One thing I'm wondering about is the process for releasing this feature – since it requires a db upgrade it could be that we want to set the published field as true for every dashboard at first so it seems that nothing has changed and everything is still visible. This seems like it might be the most accessible UX for folks at first but I'm open to other ideas. If you're in agreement I'm wondering what the best way to accomplish this would be. I'm thinking of just a simple UPDATE table statement but not sure this is the most elegant or appropriate way.

Copy link
Member

@mistercrunch mistercrunch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Tresdon just one thing, let's set the database migration to set all existing dashboards to published = True

We could also set a feature flag in front of the feature if people feel strongly about NOT enabling this feature in their environment, but I doubt this would be the case.

@mistercrunch
Copy link
Member

Hey sorry about the new conflicts, but they should be easy to solve, we just started using black, so you should be able to just run black superset/ and black tests/, and then rebase and things should go smooth. Let us know if that's not the case...

https://github.com/python/black

@Tresdon
Copy link
Contributor Author

Tresdon commented Jun 26, 2019

Zero worries, I think the formatting as well as the migration initialization are both figured out – but waiting to hear if travis has anything to say about linting. I quite like the feature flag system for the project and am willing to implement it – but I also don't foresee many folks finding that it's a feature they can't live with.

@Tresdon
Copy link
Contributor Author

Tresdon commented Jun 27, 2019

@mistercrunch Any idea how to re-run travis without changing files? The error it got seems to be a Connection reset by peer. The test failed on making a request to https.github.com and I can't seem to reproduce it locally

@Tresdon
Copy link
Contributor Author

Tresdon commented Jul 1, 2019

@mistercrunch any thoughts on the changes or does it all look good to you?

@mistercrunch mistercrunch merged commit 97ffb76 into apache:master Jul 11, 2019
@mistercrunch mistercrunch mentioned this pull request Jul 13, 2019
6 tasks
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 0.34.0 labels Feb 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels .pinned Draws attention size/L 🚢 0.34.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.