Disable fields added from unknown saved object types

[rudolf]

Summary

Fixes #67086 so that we don't need workarounds to remove old type mappings like https://github.com/aaronjcaldwell/kibana/blob/8a04349b3c5b5c6cb852cf4cef83b4bb9884e8d3/x-pack/plugins/maps/server/saved_objects/maps_telemetry.ts#L8-L23

When upgrading from 7.7.1 -> 7.8.0 -> master this PR removes 123 fields, but 122 of these are because security_solution is disabled by default in master but was enabled in 7.8 (as the siem plugin).

           "server":{
              "type":"object",
              "dynamic":"false"
           },
           "siem-detection-engine-rule-actions":{
              "type":"object",
              "dynamic":"false"
           },
           "siem-detection-engine-rule-status":{
              "type":"object",
              "dynamic":"false"
           },
           "siem-ui-timeline":{
              "type":"object",
              "dynamic":"false"
           },
           "siem-ui-timeline-note":{
              "type":"object",
              "dynamic":"false"
           },
           "siem-ui-timeline-pinned-event":{
              "type":"object",
              "dynamic":"false"
           },

Limitations

Setting dynamic: false on the top-level index mappings would remove a lot of validation provided by ES so until Saved Objects allow plugins to define their own validation that's not really an option. Since we always retain the documents belonging to unknown saved objects types, we need to keep the minimal mappings to be able to store these. This means we can only disable the fields underneath an unknown type's mappings with dynamic: false.

Although plugins could use a "core field datatype" like text for their mappings from Legacy this is unlikely and most plugins used an object data type.

I don't think this has ever been done, but should we for instance deprecate the namespace field (type: 'keyword') in favour of namespaces then this PR won't clean up that field.

It's also important to note that although we clean up all the fields of a unknown type, the top-level field for that type will still count towards the field count limit.

As an example, the following mapping:

{
  "dynamic": strict,
  "properties": {
    "a": {
      "properties": {
        "b": {
          "properties": {
            "c": {
              "type": "text"
            }
          }
        }
      }
    }
  }
}

Would add fields a, b and c that count towards the field count limit (even though only c is added to the index). We can disable everything below a but with dynamic: false but we would still be left with one field a.

So this would remove one of the two fields left over by server.uuid from #66963 and potentially leave one maps-telemetry field over from #69995 when users upgrade from older versions.

We will eventually completely solve this problem in #66056 by refusing to upgrade with unknown data so then we don't need any mappings for unknown types hanging around.

Checklist

Delete any items that are not applicable to this PR.

• Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
• Documentation was added for features that require explanation or tutorials
• Unit or functional tests were updated or added to match the most common scenarios
• This was checked for keyboard-only and screenreader accessibility
• This renders correctly on smaller devices using a responsive layout. (You can test this in your browser
• This was checked for cross-browser compatibility

For maintainers

• This was checked for breaking API changes and was labeled appropriately

[elasticmachine]

Pinging @elastic/kibana-platform (Team:Platform)

[kibanamachine]

💔 Build Failed

• continuous-integration/kibana-ci/pull-request
• Commit: 27785ba
• Pipeline Steps (look for red circles / failed steps)
• Interpreting CI Failures

Failed CI Steps

• Build X-Pack Kibana
• Execute kibana-intake

Build metrics

✅ unchanged

History

• 💚 Build #59889 succeeded 6b71b0b
• 💚 Build #59397 succeeded 1d6cfc9
• 💔 Build #59359 failed 4fd945c

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[rudolf]

Type check failure comes from master (and has since been fixed) so ignoring.