Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Detection Rules] Add updates from 7.11.2 rules #91553

Merged
merged 3 commits into from
Feb 17, 2021

Conversation

brokensound77
Copy link
Contributor

Summary

Rule updates from https://github.com/elastic/detection-rules/tree/7.11

The only update here is adding the timestamp_override field to many of the rules. Details here

The existing rules PR #91082, should merge after this is merged and backported

Checklist

@brokensound77 brokensound77 added release_note:skip Skip the PR/issue when compiling release notes v7.11.2 labels Feb 16, 2021
@brokensound77 brokensound77 requested a review from a team as a code owner February 16, 2021 19:54
@brokensound77 brokensound77 added Feature:Detection Rules Anything related to Security Solution's Detection Rules Team:Detections and Resp Security Detection Response Team labels Feb 16, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detections-response (Team:Detections and Resp)

Copy link

@bm11100 bm11100 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@spong
Copy link
Member

spong commented Feb 16, 2021

Flakiness w/ an ML test: #91450

@elasticmachine merge upstream

@spong
Copy link
Member

spong commented Feb 16, 2021

@elasticmachine merge upstream

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Copy link
Member

@spong spong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked out, tested locally, and all 356 updates were identified and was able to update without error! 🙌 Scanned rules and those changes LGTM as well.

image

In testing however, an issue was uncovered (#91594) within the Detection Engine where in some cases we're not falling back to @timestamp when the timestampOverride field isn't mapped. This was the result of a different ES error response format than expected.

We'll be fixing this for 7.11.2, however this raises the question of whether or not the Elastic Endgame rules which only point to the endgame-* index should be updated to use the event.ingested timestamp override as (afaik) it will never exist. Going to approve this PR as to not hold up the 7.12 changes pending on this, but we should re-visit not updating those rules if they are expected to always fallback to @timestamp since it's added overhead to the cluster (extra queries), and the rule will be in a partial failure/warning state saying that event.ingested is not mapped when there's nothing the user can do to resolve this issue.

@spong spong requested a review from a team February 17, 2021 05:14
@brokensound77
Copy link
Contributor Author

That sounds good to me @spong, thanks for running those errors down!

@brokensound77 brokensound77 merged commit df46bfc into elastic:master Feb 17, 2021
@brokensound77 brokensound77 deleted the rules/7.11.2 branch February 17, 2021 06:10
brokensound77 added a commit to brokensound77/kibana that referenced this pull request Feb 17, 2021
* [Detection Rules] Add 7.11.2 rules
* update timestamp_override fields for certain rules
gmmorris added a commit to gmmorris/kibana that referenced this pull request Feb 17, 2021
* master: (157 commits)
  [DOCS] Adds machine learning to the security section of alerting (elastic#91501)
  [Uptime] Ping list step screenshot caption formatting (elastic#91403)
  [Vislib] Use timestamp on brush event instead of iso dates (elastic#91483)
  [Application Usage] Remove deprecated & unused legacy.appChanged API (elastic#91464)
  Migrate logstash, monitoring, url_drilldowns, xpack_legacy to ts projects (elastic#91194)
  [APM] Wrap Elasticsearch client errors (elastic#91125)
  [APM] Fix optimize-tsconfig script (elastic#91487)
  [Discover][docs] Add searchFieldsFromSource description (elastic#90980)
  Adds support for 'ip' data type (elastic#85087)
  [Detection Rules] Add updates from 7.11.2 rules (elastic#91553)
  [SECURITY SOLUTION] Eql in timeline (elastic#90816)
  [APM] Correlations Beta (elastic#86477) (elastic#89952)
  [Security Solutions][Detection Engine] Adds a warning banner when the alerts data has not been migrated yet. (elastic#90258)
  [Security Solution] [Timeline] Endpoint row renderers (2nd batch) (elastic#91446)
  skip flaky suite (elastic#91450)
  skip flaky suite (elastic#91592)
  [Security Solution][Endpoint][Admin] Endpoint Details UX Enhancements (elastic#90870)
  [ML] Add better UI support for runtime fields Transforms  (elastic#90363)
  [Security Solution] [Detections] Replace 'partial failure' with 'warning' for rule statuses (elastic#91167)
  [Security Solution][Detections] Adds Indicator path config for indicator match rules (elastic#91260)
  ...
brokensound77 added a commit that referenced this pull request Feb 17, 2021
* [Detection Rules] Add 7.11.2 rules
* update timestamp_override fields for certain rules
brokensound77 added a commit to brokensound77/kibana that referenced this pull request Feb 17, 2021
* [Detection Rules] Add 7.11.2 rules
* update timestamp_override fields for certain rules
spong pushed a commit that referenced this pull request Feb 18, 2021
* [Detection Rules] Add 7.11.2 rules
* update timestamp_override fields for certain rules
spong pushed a commit that referenced this pull request Feb 18, 2021
#91771)

## Summary

Pulls updates from elastic/detection-rules#951. This basically reverts the changes made in #91553 for _only_ the endgame promotion rules

### Checklist


- [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Feb 18, 2021
elastic#91771)

## Summary

Pulls updates from elastic/detection-rules#951. This basically reverts the changes made in elastic#91553 for _only_ the endgame promotion rules

### Checklist


- [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)
kibanamachine added a commit that referenced this pull request Feb 18, 2021
#91771) (#91784)

## Summary

Pulls updates from elastic/detection-rules#951. This basically reverts the changes made in #91553 for _only_ the endgame promotion rules

### Checklist


- [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>
spong pushed a commit that referenced this pull request Feb 18, 2021
## Summary

Pull updates to detection rules from https://github.com/elastic/detection-rules/tree/7.12

This should not merge until after #91553 is merged and backported

### Checklist

Delete any items that are not applicable to this PR.

- [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Feb 18, 2021
## Summary

Pull updates to detection rules from https://github.com/elastic/detection-rules/tree/7.12

This should not merge until after elastic#91553 is merged and backported

### Checklist

Delete any items that are not applicable to this PR.

- [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)
kibanamachine added a commit that referenced this pull request Feb 18, 2021
## Summary

Pull updates to detection rules from https://github.com/elastic/detection-rules/tree/7.12

This should not merge until after #91553 is merged and backported

### Checklist

Delete any items that are not applicable to this PR.

- [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>
spong pushed a commit to spong/kibana that referenced this pull request Feb 18, 2021
## Summary

Pull updates to detection rules from https://github.com/elastic/detection-rules/tree/7.12

This should not merge until after elastic#91553 is merged and backported

### Checklist

Delete any items that are not applicable to this PR.

- [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)
spong added a commit that referenced this pull request Feb 18, 2021
## Summary

Pull updates to detection rules from https://github.com/elastic/detection-rules/tree/7.12

This should not merge until after #91553 is merged and backported

### Checklist

Delete any items that are not applicable to this PR.

- [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)

Co-authored-by: Justin Ibarra <brokensound77@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Detection Rules Anything related to Security Solution's Detection Rules release_note:skip Skip the PR/issue when compiling release notes Team:Detections and Resp Security Detection Response Team v7.11.2 v7.12.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants