[Logs UI] Reorganise log rate anomaly table

[Kerry350]

Summary

This PR implements #63671, and lays a lot of the foundations for #64755.

From a high-level the following changes have been made:

• A new API endpoint has been created to retrieve log entry rate log examples. This endpoint is very similar to the log category example endpoint. I didn't want to try and merge / DRY up these two endpoints too early. They do have a lot of similarities, especially at the route handler level, but the actual queries and types differ. When [Logs UI] Show category anomalies in anomaly table #64755 is implemented it may make sense to to have these converge (in places).

• The loading / no results / has failed to load results portion of the log examples display has been refactored into a reusable component - this is now being used for the log entry rate examples, and the category examples on the categories page.

• The severity indicator is also reusable now.

• The API for results currently remains log entry rate "graph centric" and not anomaly centric, and the client side just formats the data in a way that makes sense for displaying the new anomaly table format. For [Logs UI] Show category anomalies in anomaly table #64755 the API will be altered to be anomaly centric, and will return log entry rate and category anomalies in one. (This is to say that this doesn't need to be commented on for this round of work).

• There are some things "left behind" here, like the log rate graph. [Logs UI] anomaly view cleanup #65046 will handle cleaning these parts up.

• Table is now paginated as we're displaying a row per anomaly and not per dataset.

Testing

For variety it can be helpful to have data from several datasets.

[Kerry350]

@weltenwort

When using the page to inspect the anomalies I found that the table is not as useful, because it potentially contains many identically-looking rows which the can't easily associate to the point in time when the anomaly occurred:

Yeah, that's a fair point.

Implementing Show the start and/or end time of the bucket in the row. would be simple and I can't see any drawbacks to adding it.

Link the contents of the table and the chart via some interaction such as highlighting the bucket and associated table rows on click or hover. could then be done in a followup.

[Kerry350]

@weltenwort I just wanted to circle back to this:

Show the start and/or end time of the bucket in the row.

As I had some questions once I got to implementation.

In this work we alter the table that shows log rate anomalies. In #64755 we'll add category anomalies to the table. But the chart remains linked to log rate anomalies only.

If the start and / or end times of the bucket are used, that makes sense for now with log rate anomalies. But there won't be associated buckets for the category anomalies. A few questions:

• Should we just add the start and / or end time of the bucket for log rate anomalies.
• Should we show the time the anomaly occurred for category anomalies?
• Should we instead show the time the anomaly occurred for both? And forego the bucket time display in the rows?

I'm thinking across the two sets of work at this point. Also, the new anomaly API just returns all the anomalies, within the time range (future, this would have dataset filtering too). And whilst the time of the anomaly occurence would be there, it would need to be linked client side back into the bucketed data that deals with the chart-centric data (doable, of course, just wanted to highlight it).

[weltenwort]

Good point. Showing the anomaly start time (which corresponds to the ML-controlled bucketing) for the anomaly rows seems more sensible as opposed to using the chart bucketing.

[weltenwort]

But the chart remains linked to log rate anomalies only.

I guess we're still aiming for a swimlane-like chart at some point, which could then contain all kinds of anomalies.

[weltenwort]

I couldn't provoke the problem with the missing log entries anymore 👍

I left a few questions and nits below. Do you plan on adding the time column we discussed earlier in this PR or a follow-up?

[weltenwort]

Would there be a downside to using context.core.elasticsearch.legacy.client.callAsCurrentUser instead of this?

[Kerry350]

If I'm honest I'm not sure - I noticed that with the Kibana framework callWithRequest we do a little bit of preprocessing to the ES params, before calling callAsCurrentUser, things like:

 const includeFrozen = await uiSettings.client.get(UI_SETTINGS.SEARCH_INCLUDE_FROZEN);
    if (endpoint === 'msearch') {
      const maxConcurrentShardRequests = await uiSettings.client.get(
        UI_SETTINGS.COURIER_MAX_CONCURRENT_SHARD_REQUESTS
      );
      if (maxConcurrentShardRequests > 0) {
        params = { ...params, max_concurrent_shard_requests: maxConcurrentShardRequests };
      }
    }

this seemed important, so I opted to use that version. Is it safe to bypass the shard settings and so on?

[weltenwort]

That's true... so I'm not respecting that when fetching the category examples 😬 I guess we should make a search function available via the context at some point that does something similar to our framework function.

So this is technically more correct right now 👍

[Kerry350]

I guess we should make a search function available via the context at some point that does something similar to our framework function.

Yep, sounds good.

[Kerry350]

@weltenwort

I left a few questions and nits below. Do you plan on adding the time column we discussed earlier in this PR or a follow-up?

The time column in the row, and additional empty log example header to account for the context menu are coming in here (just weren't ready with the other commits earlier).

Thanks for the new feedback 👍

[Kerry350]

@weltenwort I've responded to everything (so far) now. Other than the outstanding ML link question, and the callWithRequest question.

The dataset alignment is fixed:

And I've added a sortable Start time to the rows. I've used the Kibana dateFormat setting to format this, and the default is very verbose 😬 But it's best to go with the user's settings I guess.

[kibanamachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request
• Commit: eea486c

Build metrics

@kbn/optimizer bundle module count

id	value	diff	baseline
infra	604	+1	603

History

• 💔 Build #58355 failed 390ecdd
• 💔 Build #58312 failed 1073814
• 💔 Build #58314 failed e13df39
• 💔 Build #58326 failed 5b06276
• 💔 Build #58225 failed 5b73fcc

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[weltenwort]

The page looks good and seems to work well. This is a good basis for the inclusion of category anomalies IMHO.

Yet another piece of great work! 👏