[APM] Correlations Beta

[sorenlouv]

Summary

Related design issue elastic/apm#302

Design solution

Kapture.2020-12-22.at.20.56.53.mp4

Testing

• Validate the feature against more realistic data like DPP or the chaos environment.

Technical

• Add support for metrics (example) ([APM] Correlations support metrics for latency distribution #91572)
• Make number of buckets fixed to 15
• Update the fields that are analyzed by default.
  • Backend services
    • labels
    • service.version
    • service.node.name
    • host.ip
  • JS services
    • labels
    • user_agent.name
    • user_agent.os.name
    • url.original, user fields
• Consider using logarithmic scale for distribution charts (right now a single very slow transactions will cause all other transactions to go into the first bucket) ([APM] Correlations: Display latency distribution in a logarithmic scale #91574)

Design

• Add "Traffic" metric as another option ([APM] Add Throughput tab to Correlations #90633)
• Add tab navigation and split each metric set into separate tabs; Latency, Error rate, and Traffic
• Add small paragraph summary for each tab content that introduces the different expected value of each metric set aggregration. (copy TBD)
• Add titles for each chart (size xxs)
• Filter options should have a tooltip with explanations; "Filter by value" or "Filter out value".
• Adding a filter value should close the flyout and add the value to the query bar in the previous view.
• Add i18n support for all labels
• Change main flyout button to "Explore correlations" and add icon visTagCloud (new icon pending)
• Update charts to make them consistent with the other Transaction pages
• Change the selected value chart visualization to use the euiColorAccent as opposed to the existing red.
• Update the score badge to use default and drop the code font change. Also, add proper unit formatting.
• Change the accordion button label to "Customize fields"
• Move the button below the table
• Update the threshold field with new prepend label to show it's percentiles and add help text.
• There should be an option to reset to default, if the user changes the value.
• Sort the selected field values alphabetically.
• Add help text with a link to documentation where we can explain the reason for selecting those fields.
• Add the correlations button to the overview page in addition to the transactions experience.
• If the filter next to the value is clicked, the user should navigate go to the currently selected page with the filter in the kuery bar and the correlations flyout should collapse.
• Add the Explore correlations button to the Service overview page
• Customizing the fields for correlation should allow suggestions for fields to make it easier to add new terms for the sig terms query
• Saving field customization

Section descriptions copy

Latency

What is slowing down my service? Correlations will help discover a slower performance in a particular cohort of your data. Either by host, version, or other custom fields.

Errors

Why are some transactions failing and returning errors? Correlations will help discover a possible culprit in a particular cohort of your data. Either by host, version, or other custom fields.

Throughput

What's driving the increase or decrease in throughput to my service? Correlations will help discover a potential problem in a particular cohort of your data. Either by host, version, or other custom fields.

Saving field customization

We could either save the configurations to an index or saved object depending on what we feel is best suited to the experience. Alternatively, they're not saved and only kept for the quick analysis as you're adding them, but I think users will get the impression that their selections are saved until next time, or if they go to another service or transaction.

Update We've decided to go with a local storage approach to keep the customized fields per session for the user. We might reconsider once per-user settings are enabled on spaces (#21640) that will enable even more control per user. Also, we'll keep an eye out for feedback we get from users. I've removed the task around saving fields, as they will be saved automatically in local storage.

View examples

[elasticmachine]

Pinging @elastic/apm-ui (Team:apm)

[formgeist]

@sqren @nehaduggal Updated the list of changes and additions to the feature.

[formgeist]

@sqren @nehaduggal I made some changes above to separate things from new additions to changes and a specific portion around the fields customization, which I wasn't sure how we imagine we want to implement further. Please when you have time if you can provide some feedback on this.

[formgeist]

Updated the issue description with a decision on the fields customization, which we'll make local storage enabled per user session. This is the most practical and essential saving of the fields we can imagine would be useful for the first iteration. We might reconsider when spaces also become user profile specific, which will enable users per space to define the fields. We might also get other feedback from users once the feature is released, so we should keep an eye out for that.

cc @nehaduggal @sqren

[ogupte]

We should also take this opportunity to rename files, components, and data type to reflect the current feature name "Correlations" instead of the old feature name "Significant terms" (#89028)

[sorenlouv]

@ogupte I added a new section "Technical" with some more items. LMK if you have questions about it.

[formgeist]

@ogupte I've added another task to add the "Explore correlations" button to invoke the flyout to the Service overview page as well ^ cc @sqren @nehaduggal

[formgeist]

@ogupte @nehaduggal Added some quick descriptions for each section in the flyouts to describe what use each of them are to the investigation. Let me know if you have any improvements to the copy? I wanted to keep a similar theme to them.