Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SIEM][Detections] Add rule.{authors, license} to the schema #66317

Closed
rw-access opened this issue May 12, 2020 · 2 comments
Closed

[SIEM][Detections] Add rule.{authors, license} to the schema #66317

rw-access opened this issue May 12, 2020 · 2 comments
Assignees
@spong
Labels
enhancement New value added to drive a business result Feature:Detection Rules Anything related to Security Solution's Detection Rules Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:SIEM v7.9.0

Comments

@rw-access
Copy link
Contributor

rw-access commented May 12, 2020 •
edited
Loading

Relates to #65941

Describe the feature:
We need a way to keep track of detection rules with multiple authors or contributors. I've thrown around the idea of "authors" vs "contributors." The concept of contributors could make more sense, and getting rid of the concept of authorship could make the semantics simpler. For the EQL Analytics Library, we only had the notion of contributors, but I'm flexible and not overly attached to the semantics behind these words.

ECS currently has a field called rule.author which is an array. I don't personally know the interplay between a rule and the signals that it creates, so I don't know if this field makes perfect sense in all contexts. I'll defer to the SIEM team to think that through more thoroughly.

For rules provided by the Elastic Intelligence & Analytics team, these will only have one author/contributor. But if a user decided to fork the rule, they could add themselves to the list. I think that this should be exposed to the UI as well, and not be buried in the API.

License:
(I think) We also need to add a field that contain the license that corresponds to a rule. This should be accessible from the API and the UI. I think both Elastic provided rules and user provided rules will use this field. I don't think it should be strictly required, at least for user created rules.

Update: We only allow licenses that can be sublicensed as Basic. I think Elastic license works for Elastic created detections, as long as we keep the notices around. I think the field won't need to be populated for rules we provide. But users may still share rules with each other that are licensed differently.

Describe a specific use case for the feature:
Many rules tend to be changed by multiple people across time, and in general the infosec community can be very collaborative on detection logic.
@rw-access rw-access added Team:SIEM Feature:Detection Rules Anything related to Security Solution's Detection Rules labels May 12, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/siem (Team:SIEM)

@spong spong added enhancement New value added to drive a business result v7.9.0 labels May 12, 2020
@rw-access rw-access changed the title [SIEM][Detections] Add rule author(s) field to the schema [SIEM][Detections] Add rule.{authors, license} to the schema May 13, 2020
@spong spong mentioned this issue Jun 30, 2020
Merged
5 tasks
spong added a commit that referenced this issue Jul 2, 2020
@spong
[Security] Adds field mapping support to rule creation (#70288)
591e103 
## Summary

Resolves: #65941, #66317, and `Add support for "building block" alerts`

This PR is `Part I` and adds additional fields to the `rules schema` in supporting the ability to map and override fields when generating alerts. A few bookkeeping fields like `license` and `author` have been added as well. The new fields are as follows:

``` ts
export interface TheseAreTheNewFields {
  author: string[];
  building_block_type: string; // 'default'
  license: string;
  risk_score_mapping: Array<
    {
      field: string;
      operator: string; // 'equals'
      value: string;
    }
  >;
  rule_name_override: string;
  severity_mapping: Array<
    {
      field: string;
      operator: string; // 'equals'
      value: string;
      severity: string; // 'low' | 'medium' | 'high' | 'critical'
    }
  >;
  timestamp_override: string;
}
```

These new fields are exposed as additional settings on the `About rule` section of the Rule Creation UI.

##### Default collapsed view, no severity or risk score override specified:
<p align="center">
  <img width="500" src="https://user-images.githubusercontent.com/2946766/86090417-49c0ee80-ba67-11ea-898f-a43af6d9383f.png" />
</p>

##### Severity & risk score override specified:
<p align="center">
  <img width="500" src="https://user-images.githubusercontent.com/2946766/86091165-a8d33300-ba68-11ea-86ac-89393a7ca3f5.png" />
</p>

##### Additional fields in Advanced settings:
<p align="center">
  <img width="500" src="https://user-images.githubusercontent.com/2946766/86091256-cbfde280-ba68-11ea-9b63-acf2524039bd.png" />
</p>


Note: This PR adds the fields to the `Rules Schema`, the `signals index mapping`,  and creates the UI for adding these fields during Rule Creation/Editing. The follow-up `Part II` will add the business logic for mapping fields during `rule execution`, and also add UI validation/additional tests.

### Checklist

Delete any items that are not applicable to this PR.

- [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)
- [ ] [Documentation](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials
  - Syncing w/ @benskelker 
- [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios
- [x] This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)

### For maintainers

- [x] This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)
FrankHassanabad pushed a commit to FrankHassanabad/kibana that referenced this issue Jul 2, 2020
@spong @FrankHassanabad
[Security] Adds field mapping support to rule creation (elastic#70288)
c9f5867 
## Summary

Resolves: elastic#65941, elastic#66317, and `Add support for "building block" alerts`

This PR is `Part I` and adds additional fields to the `rules schema` in supporting the ability to map and override fields when generating alerts. A few bookkeeping fields like `license` and `author` have been added as well. The new fields are as follows:

``` ts
export interface TheseAreTheNewFields {
  author: string[];
  building_block_type: string; // 'default'
  license: string;
  risk_score_mapping: Array<
    {
      field: string;
      operator: string; // 'equals'
      value: string;
    }
  >;
  rule_name_override: string;
  severity_mapping: Array<
    {
      field: string;
      operator: string; // 'equals'
      value: string;
      severity: string; // 'low' | 'medium' | 'high' | 'critical'
    }
  >;
  timestamp_override: string;
}
```

These new fields are exposed as additional settings on the `About rule` section of the Rule Creation UI.

##### Default collapsed view, no severity or risk score override specified:
<p align="center">
  <img width="500" src="https://user-images.githubusercontent.com/2946766/86090417-49c0ee80-ba67-11ea-898f-a43af6d9383f.png" />
</p>

##### Severity & risk score override specified:
<p align="center">
  <img width="500" src="https://user-images.githubusercontent.com/2946766/86091165-a8d33300-ba68-11ea-86ac-89393a7ca3f5.png" />
</p>

##### Additional fields in Advanced settings:
<p align="center">
  <img width="500" src="https://user-images.githubusercontent.com/2946766/86091256-cbfde280-ba68-11ea-9b63-acf2524039bd.png" />
</p>


Note: This PR adds the fields to the `Rules Schema`, the `signals index mapping`,  and creates the UI for adding these fields during Rule Creation/Editing. The follow-up `Part II` will add the business logic for mapping fields during `rule execution`, and also add UI validation/additional tests.

### Checklist

Delete any items that are not applicable to this PR.

- [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)
- [ ] [Documentation](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials
  - Syncing w/ @benskelker 
- [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios
- [x] This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)

### For maintainers

- [x] This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)
FrankHassanabad added a commit that referenced this issue Jul 2, 2020
@FrankHassanabad @spong
[7.x] [Security] Adds field mapping support to rule creation (#70288) (…
00eed1b 
…#70603)

* [Security] Adds field mapping support to rule creation (#70288)

## Summary

Resolves: #65941, #66317, and `Add support for "building block" alerts`

This PR is `Part I` and adds additional fields to the `rules schema` in supporting the ability to map and override fields when generating alerts. A few bookkeeping fields like `license` and `author` have been added as well. The new fields are as follows:

``` ts
export interface TheseAreTheNewFields {
  author: string[];
  building_block_type: string; // 'default'
  license: string;
  risk_score_mapping: Array<
    {
      field: string;
      operator: string; // 'equals'
      value: string;
    }
  >;
  rule_name_override: string;
  severity_mapping: Array<
    {
      field: string;
      operator: string; // 'equals'
      value: string;
      severity: string; // 'low' | 'medium' | 'high' | 'critical'
    }
  >;
  timestamp_override: string;
}
```

These new fields are exposed as additional settings on the `About rule` section of the Rule Creation UI.

##### Default collapsed view, no severity or risk score override specified:
<p align="center">
  <img width="500" src="https://user-images.githubusercontent.com/2946766/86090417-49c0ee80-ba67-11ea-898f-a43af6d9383f.png" />
</p>

##### Severity & risk score override specified:
<p align="center">
  <img width="500" src="https://user-images.githubusercontent.com/2946766/86091165-a8d33300-ba68-11ea-86ac-89393a7ca3f5.png" />
</p>

##### Additional fields in Advanced settings:
<p align="center">
  <img width="500" src="https://user-images.githubusercontent.com/2946766/86091256-cbfde280-ba68-11ea-9b63-acf2524039bd.png" />
</p>


Note: This PR adds the fields to the `Rules Schema`, the `signals index mapping`,  and creates the UI for adding these fields during Rule Creation/Editing. The follow-up `Part II` will add the business logic for mapping fields during `rule execution`, and also add UI validation/additional tests.

### Checklist

Delete any items that are not applicable to this PR.

- [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)
- [ ] [Documentation](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials
  - Syncing w/ @benskelker 
- [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios
- [x] This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)

### For maintainers

- [x] This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)

* Fixed i18n keys

Co-authored-by: Garrett Spong <spong@users.noreply.github.com>
@spong
Copy link
Member

spong commented Jul 6, 2020

These fields were added in #70288, closing... 🙂

@spong spong closed this as completed Jul 6, 2020
@spong spong mentioned this issue Jul 8, 2020
Closed
@MindyRS MindyRS added the Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. label Oct 27, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@spong spong

Labels
enhancement New value added to drive a business result Feature:Detection Rules Anything related to Security Solution's Detection Rules Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:SIEM v7.9.0
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@MindyRS @spong @elasticmachine @rw-access