[Ingest] Data streams page UI

[jen-huang]

View in Figma

Build the Data streams page which shows a table of data streams. Data stream detail page is out of scope for alpha 1.

Tasks

• Create data streams tab and header
• Add table of indices

[elasticmachine]

Pinging @elastic/ingest-management (Feature:EPM)

[ruflin]

@hbharding Any progress on this one? I think to simplify things we can leave out package, size, age and last activity in the table above and then keep iterating on it to see which info we can add further.

[hbharding]

I'll be working on this one today

[hbharding]

I updated the data stream design based on Figma comments and past discussions. If the level of effort is too great, we can remove the integration, last activity, size, shards, replicas, and actions columns. We'll be left with a pretty sparse table, but that's probably fine for alpha 1. Let me know and I can update the design no problem.

In the past, we've discussed the need for the user to be able to edit the ILM Policy or to view the data in a dashboard. I thought this would be a logical place to do so. Here's a screenshot of what that could look like... the list of actions is sample of what we could do, so I'm posting mostly for discussion. We may not need any actions for alpha 1. @ruflin?

[ruflin]

++ on linking from here to Dashboards. Also ILM will fit well here.

There will also be data streams without integrations. I wonder if we need an icon / name for this?

[mostlyjason]

@hbharding it still says data sets in a few places. I thought we were going to call these data streams?

[hbharding]

Originally the first column said "data streams", with values like [type]-[namespace]-[dataset] (i.e. "logs-prod-system.cpu"), but @ruflin commented in Figma that it would make sense to separate these into their own columns. This allows you to sort the order of the table by those columns, and it also makes it more apparent which fields you can filter by. It also makes the table more legible.

But, I get why this is confusing. It's introducing yet another term, "data set", and we're on the "data streams" page. As i see it, we could:

1. Condense the 3 columns into a single column called "data streams", with values [type]-[namespace]-[dataset] (as I had it previously)
2. Rename the page to say "Data sets" or perhaps just "Data" and keep the table as shown in the screenshot.
3. Open to other ideas :)

[ruflin]

I still think what you have above in the screenshot is correct. It is the datastreams table, each line is a data stream. But there are additional infos around data streams we share here, like which dataset it comes from, which type it is and why namespace. I would assume, these 3 properties is also what users will sort the table on. Get me all data streams for the system.cpu dataset.

I definitively see the point around the 3 terms we introduce here that users will have to learn. At the same time it is super important that any user who wants to query and filter data in the dashboards, understands these 3 terms to efficiently query the data.

Suggestion: Lets get this in for alpha1 as above and iterate on it. I have a few ideas on how we could improve it:

• Group data streams by package
• Only show some of the info in detail page
• Others :-)

[hbharding]

Awesome, thanks Nicolas. Yeah, lets go as-is for Alpha. It's a solid first step from which we can iterate.