[Custom threshold] Executor does not respect query:allowLeadingWildcards advance setting #189072
Assignees
Labels
bug
Fixes for quality problems that affect the customer experience
Feature: Custom threshold
Observability custom threshold rule type
Team:obs-ux-management
Observability Management User Experience Team
Comments
maryam-saeidi
added
bug
|
Pinging @elastic/obs-ux-management-team (Team:obs-ux-management) |
maryam-saeidi
added a commit
that referenced
this issue
Jul 31, 2024
…ery filter (#189488) Partially fixes. #189072 ## Summary In this PR, we pass the `query:allowLeadingWildcards` for the optional filter to the custom threshold (specifically `getSearchConfigurationBoolQuery` function that generates the related ES Query). |Before|After| |----|---| ||| #### Rule <img src="https://github.com/user-attachments/assets/70d2de37-2285-450f-88bf-45aa88954019" width=500 /> --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
maryam-saeidi
added a commit
to maryam-saeidi/kibana
that referenced
this issue
Aug 1, 2024
…ery filter (elastic#189488) Partially fixes. elastic#189072 In this PR, we pass the `query:allowLeadingWildcards` for the optional filter to the custom threshold (specifically `getSearchConfigurationBoolQuery` function that generates the related ES Query). |Before|After| |----|---| ||| <img src="https://github.com/user-attachments/assets/70d2de37-2285-450f-88bf-45aa88954019" width=500 /> --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> (cherry picked from commit 5219a1f)
maryam-saeidi
added a commit
that referenced
this issue
Aug 2, 2024
…onal query filter (#189488) (#189705) # Backport This will backport the following commits from `main` to `8.15`: - [[Custom threshold] Respect query:allowLeadingWildcards in optional query filter (#189488)](#189488) <!--- Backport version: 8.9.8 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Maryam Saeidi","email":"maryam.saeidi@elastic.co"},"sourceCommit":{"committedDate":"2024-07-31T13:29:04Z","message":"[Custom threshold] Respect query:allowLeadingWildcards in optional query filter (#189488)\n\nPartially fixes. #189072\r\n\r\n## Summary\r\n\r\nIn this PR, we pass the `query:allowLeadingWildcards` for the optional\r\nfilter to the custom threshold (specifically\r\n`getSearchConfigurationBoolQuery` function that generates the related ES\r\nQuery).\r\n\r\n|Before|After|\r\n|----|---|\r\n\r\n|||\r\n\r\n#### Rule\r\n\r\n<img\r\nsrc=\"https://github.com/user-attachments/assets/70d2de37-2285-450f-88bf-45aa88954019\"\r\nwidth=500 />\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"5219a1f14d1af812624282e59170503d8a071bd4","branchLabelMapping":{"^v8.16.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["bug","release_note:fix","backport:prev-minor","ci:project-deploy-observability","Team:obs-ux-management","Feature: Custom threshold","v8.16.0"],"number":189488,"url":"https://github.com/elastic/kibana/pull/189488","mergeCommit":{"message":"[Custom threshold] Respect query:allowLeadingWildcards in optional query filter (#189488)\n\nPartially fixes. #189072\r\n\r\n## Summary\r\n\r\nIn this PR, we pass the `query:allowLeadingWildcards` for the optional\r\nfilter to the custom threshold (specifically\r\n`getSearchConfigurationBoolQuery` function that generates the related ES\r\nQuery).\r\n\r\n|Before|After|\r\n|----|---|\r\n\r\n|||\r\n\r\n#### Rule\r\n\r\n<img\r\nsrc=\"https://github.com/user-attachments/assets/70d2de37-2285-450f-88bf-45aa88954019\"\r\nwidth=500 />\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"5219a1f14d1af812624282e59170503d8a071bd4"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.16.0","labelRegex":"^v8.16.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/189488","number":189488,"mergeCommit":{"message":"[Custom threshold] Respect query:allowLeadingWildcards in optional query filter (#189488)\n\nPartially fixes. #189072\r\n\r\n## Summary\r\n\r\nIn this PR, we pass the `query:allowLeadingWildcards` for the optional\r\nfilter to the custom threshold (specifically\r\n`getSearchConfigurationBoolQuery` function that generates the related ES\r\nQuery).\r\n\r\n|Before|After|\r\n|----|---|\r\n\r\n|||\r\n\r\n#### Rule\r\n\r\n<img\r\nsrc=\"https://github.com/user-attachments/assets/70d2de37-2285-450f-88bf-45aa88954019\"\r\nwidth=500 />\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"5219a1f14d1af812624282e59170503d8a071bd4"}}]}] BACKPORT-->
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Aug 14, 2024
…QL validation in the custom threshold rule API param validation (elastic#190031) Fixes elastic#189072 Related elastic#190029 ## Summary This PR updates the KQL validation on the server side by passing the Kibana leadingWildcard setting as true during validation. This means that even if this configuration is disabled in Kibana, we will still allow saving such a filter in the rule, but it will fail during rule execution. I've created a separate ticket to discuss how to apply the KQL validation correctly during API param validation. ([issue](elastic#190029)) This fix will solve the following issues: <img src="https://github.com/user-attachments/assets/d99177cb-d4cd-4f33-9a60-8575d87372e3" width=500 /> We also have [proper validation on the UI side](https://github.com/elastic/kibana/blob/main/x-pack/plugins/observability_solution/observability/public/components/custom_threshold/components/validation.tsx#L60,L64) that considers Kibana setting during validation: |Error|Leading wildcard error| |---|---| | || (cherry picked from commit ae4d522)
maryam-saeidi
added a commit
that referenced
this issue
Aug 15, 2024
…o the KQL validation in the custom threshold rule API param validation (#190031) (#190494) # Backport This will backport the following commits from `main` to `8.15`: - [[Custom threshold] Always pass allowLeadingWildcards as true to the KQL validation in the custom threshold rule API param validation (#190031)](#190031) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Maryam Saeidi","email":"maryam.saeidi@elastic.co"},"sourceCommit":{"committedDate":"2024-08-14T10:38:53Z","message":"[Custom threshold] Always pass allowLeadingWildcards as true to the KQL validation in the custom threshold rule API param validation (#190031)\n\nFixes #189072\r\nRelated #190029\r\n\r\n## Summary\r\n\r\nThis PR updates the KQL validation on the server side by passing the\r\nKibana leadingWildcard setting as true during validation. This means\r\nthat even if this configuration is disabled in Kibana, we will still\r\nallow saving such a filter in the rule, but it will fail during rule\r\nexecution.\r\nI've created a separate ticket to discuss how to apply the KQL\r\nvalidation correctly during API param validation.\r\n([issue](https://github.com/elastic/kibana/issues/190029))\r\n\r\nThis fix will solve the following issues:\r\n<img\r\nsrc=\"https://github.com/user-attachments/assets/d99177cb-d4cd-4f33-9a60-8575d87372e3\"\r\nwidth=500 />\r\n\r\n\r\nWe also have [proper validation on the UI\r\nside](https://github.com/elastic/kibana/blob/main/x-pack/plugins/observability_solution/observability/public/components/custom_threshold/components/validation.tsx#L60,L64)\r\nthat considers Kibana setting during validation:\r\n\r\n|Error|Leading wildcard error|\r\n|---|---|\r\n|\r\n||","sha":"ae4d522b52b2c3573c4e276bfd38ecec00d9ff96","branchLabelMapping":{"^v8.16.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["bug","release_note:fix","backport:prev-minor","ci:project-deploy-observability","Team:obs-ux-management","Feature: Custom threshold","v8.16.0"],"title":"[Custom threshold] Always pass allowLeadingWildcards as true to the KQL validation in the custom threshold rule API param validation","number":190031,"url":"https://github.com/elastic/kibana/pull/190031","mergeCommit":{"message":"[Custom threshold] Always pass allowLeadingWildcards as true to the KQL validation in the custom threshold rule API param validation (#190031)\n\nFixes #189072\r\nRelated #190029\r\n\r\n## Summary\r\n\r\nThis PR updates the KQL validation on the server side by passing the\r\nKibana leadingWildcard setting as true during validation. This means\r\nthat even if this configuration is disabled in Kibana, we will still\r\nallow saving such a filter in the rule, but it will fail during rule\r\nexecution.\r\nI've created a separate ticket to discuss how to apply the KQL\r\nvalidation correctly during API param validation.\r\n([issue](https://github.com/elastic/kibana/issues/190029))\r\n\r\nThis fix will solve the following issues:\r\n<img\r\nsrc=\"https://github.com/user-attachments/assets/d99177cb-d4cd-4f33-9a60-8575d87372e3\"\r\nwidth=500 />\r\n\r\n\r\nWe also have [proper validation on the UI\r\nside](https://github.com/elastic/kibana/blob/main/x-pack/plugins/observability_solution/observability/public/components/custom_threshold/components/validation.tsx#L60,L64)\r\nthat considers Kibana setting during validation:\r\n\r\n|Error|Leading wildcard error|\r\n|---|---|\r\n|\r\n||","sha":"ae4d522b52b2c3573c4e276bfd38ecec00d9ff96"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/190031","number":190031,"mergeCommit":{"message":"[Custom threshold] Always pass allowLeadingWildcards as true to the KQL validation in the custom threshold rule API param validation (#190031)\n\nFixes #189072\r\nRelated #190029\r\n\r\n## Summary\r\n\r\nThis PR updates the KQL validation on the server side by passing the\r\nKibana leadingWildcard setting as true during validation. This means\r\nthat even if this configuration is disabled in Kibana, we will still\r\nallow saving such a filter in the rule, but it will fail during rule\r\nexecution.\r\nI've created a separate ticket to discuss how to apply the KQL\r\nvalidation correctly during API param validation.\r\n([issue](https://github.com/elastic/kibana/issues/190029))\r\n\r\nThis fix will solve the following issues:\r\n<img\r\nsrc=\"https://github.com/user-attachments/assets/d99177cb-d4cd-4f33-9a60-8575d87372e3\"\r\nwidth=500 />\r\n\r\n\r\nWe also have [proper validation on the UI\r\nside](https://github.com/elastic/kibana/blob/main/x-pack/plugins/observability_solution/observability/public/components/custom_threshold/components/validation.tsx#L60,L64)\r\nthat considers Kibana setting during validation:\r\n\r\n|Error|Leading wildcard error|\r\n|---|---|\r\n|\r\n||","sha":"ae4d522b52b2c3573c4e276bfd38ecec00d9ff96"}}]}] BACKPORT--> --------- Co-authored-by: Maryam Saeidi <maryam.saeidi@elastic.co>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Summary
When a leading wildcard is used in an optional filter, it will not be considered correctly during rule execution.
For example, in the following scenario, the alert is fired with an observed value equal to 15, which is the total number of documents instead of 3 as shown in the chart:
Also, using the same filter in the count aggregation, KQL will generate the following error:
Acceptance criteria
The text was updated successfully, but these errors were encountered: