[Security Solution] .caseless fields are missing in .siem-signals mapping

[dplumlee]

Overview

There are no .caseless fields currently present in the .siem-signals mapping. This has already caused some issues dealing with exceptions but has the potential to cause other broken queries that won't work against the .siem-signals index

Specific discovery case

Currently the host.os.name.caseless is not mapped within .siem-signals index. When creating or updating an endpoint exception, if a user checks the bulk close option, the query does not perform as expected. It should filter by host.os.type or host.os.name.caseless, but some alerts do not contain the os.type field. If that is the case, the query and expected output will silently fail as no alerts will be found because the query is sent to the .siem-signals index which doesn't contain the caseless field mapping.

In order to reproduce:

1. Activate the Endpoint Security prebuilt rule
2. Generate an alert without an os.type field (pre 7.15 endpoint version)
3. Create an endpoint exception with the "bulk close" option checked
4. Expected result is that all alerts falling under the specified criteria should be closed but nothing occurs due to the missing mapping in .siem-signals

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[FrankHassanabad]

Is the caseless supposed to be supported by ECS?

Also what about this issue?
https://github.com/elastic/endpoint-dev/issues/9403

It looked like the endpoint uses different data driven values vs. using the caseless version from ECS that already exists

[dplumlee]

@FrankHassanabad caseless isn't supported in ECS, it's something we inject. I think it's a very similar issue to the one you linked. When triaging the specific case listed above, we found the exceptions filter had both the type and caseless fields but since it was POSTing to the .siem-signals, the caseless inclusion doesn't work for closing existing signals as its not present in the mapping. Were the type not added to the mapping either, neither filter would be valid for that exceptions case

[MadameSheema]

@deepikakeshav-qasource please validate the fix of this issue on the current BC. Thanks

[ghost]

Hi @MadameSheema ,

We have validated this ticket on 7.15.0 BC6 build and Please find the below observations:

Build Details:

Version:7.15.0 BC6
Commit:ab43f574e8ea01f5093fe92b2f51c4b686eb6e63
Build:44026

Observations:

• Able to see the alerts when add the filter host.os.type:exist

• Able to Close all alerts that match this exception and were generated by this rule when add the fieldhost.os.name.caseless in endpoint exception modal

host.os.name.caseless_close.all.mp4

• Able to Close all alerts that match this exception and were generated by this rule when add the field host.os.name.caseless in Rule exception modal

host.os.name.caseless_close.all_rule_exception.mp4

• Close this alert functionality is working when create the exception with host.os.name.caseless for endpoint exception and Rule exception.

close_this_alert1.mp4

• host.os.type field is not displayed on the Endpoint exceptions and Rule exception modal.

Endpoint Exception

Rule Exception:

Please let us know if we are missing anything or anything else is required to be tested.

Thanks!!

[MadameSheema]

As per the comments on #111455 the results are expected. Thanks @deepikakeshav-qasource