Metricbeat kafka dashboard

[jsoriano]

Migration of Kafka dashboard in https://demo.elastic.co to metricbeat.

Depends on #8504

[jsoriano]

Screenshot of current dashboard, main changes with respect to dashboard in demo.elastic.co:

• Removed the topic lag gauges, they looked a bit confusing to me (e.g. what to chose for the maximum value? what does the value represent?), it also didn't scale very well then there are lots of topics. We might think to replace it with a line graph.
• Added a table with info about clients, not sure if useful with big deployments, but I think it can be useful when filtering by topic and/or partition.
• Renamed some labels, from "leader partitions" to "partitions", and from "partitions" to "replicas". Number of partition leaders should be equal to number of partitions, and replicas is used to count replicas.
• Removed offsets in broker tables, they can be a bit random with multiple topics and they still appear in other visualizations.
• Reorganization of visualizations, giving more importance to consumer lag and overview info, and less importance to partition reassignments, I don't know if this is so frequent.

Something like this could be shipped for 6.5.0, but his will probably need more iterations, I'm not sure about the accuracy of some queries (specially counting replicas). And we could probably benefit of having specific topic/partition and consumers dashboards.

[jsoriano]

Added also a partition selector to dashboard controls that doesn't appear in previous screenshot.

[jsoriano]

Some more changes:

• Use of TSVB for broker and topic tables
• Number of replicas in brokers table is correct now
• Topic table shows now more topics and number of consumers per topic

I have a doubt now, I can do some things with TSVB tables that I cannot do with normal tables, but also I cannot group by multiple fields with TSVB, what I need for the consumer groups table. So I have the two kind of tables in the same dashboard. Both tables are different in style and features, so I'm not sure if continue like this, what gives in my opinion more valuable info, or show only a subset of info with consistent table styles.

The dashboard is like this now:

[jsoriano]

Fixed consistency of capitalization in table headers, and removed "Number of" from them, so there is more space for data.
We will go on with the different tables.
I'd say it is ready to merge.

[ruflin]

LGTM. Didn't test locally as no Kafka setup at the moment. Would be good to sync up with Dale when this is in.

[ruflin]

Failure is not related.

[jsoriano]

I haven't added a changelog entry, added now.

[ruflin]

WFG

[jsoriano]

@gingerwizard everything is on master now, it'd be awesome if you could give it a try. Thanks!