[Metrics UI] Metric threshold rule type: fix group by + 0 check #111772
Labels
bug
Fixes for quality problems that affect the customer experience
Feature:Alerting
Team:Infra Monitoring UI - DEPRECATED
DEPRECATED - Label for the Infra Monitoring UI team. Use Team:obs-ux-infra_services
v7.15.1
v7.16.0
Related: #76511
Summary
When a Metric Threshold rule makes a comparison of "less than" / "less than or equal to" along with setting the "Alert per" setting, it creates a broken rule that will miss real alert scenarios. The reason for this is that when the number is actually 0 because there are no documents for a given group, we no longer have that group in the data, so we can't schedule actions for that alert.
AC: We either need to disable the ability to create this kind of alert and document why it's not allowed, or determine a way we can support this functionality in a reliable and performant way.
Notes
One option considered is to store the groups that the rule sees in the persisted rule state and use that state when considering which groups should have alerts triggered. If a previously seen group no longer appears in the data, we can trigger either a 0 doc alert or a "no data" alert, depending on how we choose to handle it. Complications with this approach include:
POC PR that assumes these are "no data" alerts:
The text was updated successfully, but these errors were encountered: