[Security Solution] Add host.os.name.caseless mapping and runtime field

[marshallmain]

Summary

Closes #110130

Elastic Endpoint only recently started populating host.os.type, and instead was populating host.os.name. The logs-endpoint indices had host.os.name.caseless as a multifield on host.os.name with a lowercase normalizer. Endpoint exceptions were recently fixed to add a filter based on the os_type (linux | macos | windows), and that filter checks both host.os.type and host.os.name.caseless for the os_type to ensure it works against old Endpoint alerts that don't populate host.os.type and new alerts that do.

However, host.os.name.caseless was not mapped in the signals indices so the Close all alerts matching this exception checkbox will not close signals based on Endpoint alerts that don't populate host.os.type. These signals could be existing signals from before 7.15, or they could be signals based on new alerts coming in from endpoints that haven't been upgraded to 7.15 yet. This PR fixes the issue for existing signals by creating a runtime field host.os.name.caseless that normalizes the host.os.name field, and also adding the host.os.name.caseless normalized multi-field to the signals index mapping so newly created signals will have it as well.

Since runtime fields are slower than regular fields we also want to avoid adding them unnecessarily. This PR builds the appropriate mappings for each index version to make them backwards compatible. The runtime fields will only be added for indices whose version is <= 45 (released in 7.14), since the next release will contain host.os.name.caseless as an indexed multifield for newly written signals.

There are 2 version fields on each .siem-signals index, which can be somewhat confusing.

version is defined when the index is created and should never change. This tells us what the original mapping was on the index so we can apply the appropriate migrations/backwards compatibility mappings. This also tells us when we need to rollover and create a new index because the "regular" field mappings have changed.

aliases_version is also defined when the index is created but can change over time. This field represents whether or not an index has the latest set of aliases and runtime fields used for backwards compatibility. When we apply aliases and runtime fields to an index, this version number is updated to reflect the latest version in the source code. Even if there are no aliases or runtime fields defined for a particular version (the static version above), we still update aliases_version on all existing indices to mark that they have been checked and brought up to date.

[marshallmain]

@elasticmachine merge upstream

[dplumlee]

Do we wanna move this out into its own json file or const just so we can add future fields without mixing all the data and logic together?

[marshallmain]

We shouldn't be adding any future fields within this object, any future migrations would be separate objects at the top level of this array. And right now the field aliases are built dynamically from another JSON file, so we'd have to rework that logic to create the aliases at build time. It could be nice, but right now it's covered with a unit test against a snapshot.

[dplumlee]

is there a central point we can pull this number from so we don't have to change it piecewise every update?

[marshallmain]

This number should never change, as any future indices won't need this migration. The next migration will be a separate object and may have a different maxVersion, but we'll leave this migration the same.

[kibanamachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request
• Commit: 2bf1c2a
• Storybooks Preview

Metrics [docs]

✅ unchanged

History

• 💔 Build #152079 failed 0204df3
• 💔 Build #151629 failed 561ff2f

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[dplumlee]

Seems to work against master and fixes the closing logic for alerts that don't have a host.os.type field 👍

[MadameSheema]

As this PR is fixing a possible blocker issue, I did a bit of exploratory testing on it. I would deeply appreciate it if you can check my results since I'm not familiar with Endpoint alerts/exceptions so I'm not sure if the results are the expected ones or not:

• Case 1:

{
   "@timestamp":"2021-09-10T10:47:06.527Z",
   "event":{
      "kind":"alert",
      "module":"endpoint",
      "ingested": "2021-09-10T10:47:06.527Z"
   },
   "host":{
      "os":{
         "name":"MACOS"
      }
   }
}

Result:

• The value was properly populated on the Endpoint exceptions under host.os.name.caseless in lower-case.

• The alert was properly closed when selecting Close all alerts that match this exception and were generated by this rule

• Also as a rule exception

• Case 2:

   "@timestamp":"2021-09-10T13:37:06.527Z",
   "event":{
      "kind":"alert",
      "module":"endpoint",
      "ingested":"2021-09-10T13:37:06.527Z"
   },
   "host":{
      "os":{
         "type":"macos"
      }
   }
}

Result:

• The field host.os.type is not displayed on the Endpoint exceptions modal.

• The alert was properly closed when selecting Close all alerts that match this exception and were generated by this rule

• Also as a rule exception

• Case 3:

  "@timestamp": "2021-09-10T13:47:06.527Z",
  "event": {
    "kind": "alert",
    "module": "endpoint",
    "ingested": "2021-09-10T13:47:06.527Z"
  }
}

Result:

• The alert it is not closed when closed when selecting Close all alerts that match this exception and were generated by this rule on the Endpoint exceptions modal
• It is closed as a Rule exception

[marshallmain]

Thanks for testing it @MadameSheema! Those results are what I would expect.

[kibanamachine]

💚 Backport successful

Status	Branch	Result
✅	7.15
✅	7.x

The backport PRs will be merged automatically after passing CI.