[core.logging] Add ops logs to the KP logging system

[TinaHeiligers]

Resolves #86162

Summary

Logs ops metrics data from the Kibana platform using the core logger for feature parity with the legacy logging server.
The legacy logging server relies on the @hapi/good ops event for automatically logging ops metrics.

The ops metrics are logged from within the metrics_service, with a message format that mimics the legacy logging format (e.g: memory: 221.7MB uptime: 0:10:40 load: [8.06,7.07,13.33] delay: 0.259).
In addition, we're added ECS-compatible structured LogMeta, and used custom fields for data that isn't (yet) a standard ECS field.

The ops metrics are logged out when the ops metrics refresh (i.e. at the configured ops.interval).

Changes

• Adds logging to metrics server.
• Adds a utility for extracting the ops data.
• Adds config deprecations for logging.events.ops & updates docs.
• Disables legacy ops logs when logging.verbose: true, as this results in duplicate data with the new metrics logs.

Plugin API Changes

We are deprecating the legacy response logs which were enabled when logging.verbose: true or when using logging.events.ops. It will be removed completely in 8.0, and has been replaced with new ops logs that is provided under the metrics.ops context at the debug level:

Before

logging:
  events:
    ops: "*"

After

logging:
  loggers:
    - context: metrics.ops
      appenders: [console]
      level: debug

For more information, check out the section on logging config migration in the logging README.

How to test this:

1. Add the following logging configuration to your kibana.yml file:

#kibana.yml or kibana.dev.yml
logging:
  events:
    log: ['debug']
    json: false
    verbose: true

  appenders:
    console:
      kind: console
      layout:
        kind: pattern
        highlight: true

  root:
    appenders: [default]
    level: warn

  loggers:
    - context: metrics.metrics
      appenders: [console]
      level: debug

2. Start es and kibana.
3. Observe that the ops metrics are logged out (std out): e.g.

[2021-01-20T22:30:06.974Z][DEBUG][metrics.ops]{"ecs":{"version":"1.7.0"},"kind":"metric","category":["process","host"],"process":{"uptime":640,"memory":{"heap":{"usedInBytes":232472872}},"eventLoopDelay":0.25925004482269287},"host":{"os":{"load":{"1m":8.0625,"5m":7.07470703125,"15m":13.32568359375}}}} memory: 221.7MB uptime: 0:10:40 load: [8.06,7.07,13.33] delay: 0.259

Checklist

Delete any items that are not applicable to this PR.

• Documentation was added for features that require explanation or tutorials
• Unit or functional tests were updated or added to match the most common scenarios
• Integration tests were updated or added to match the most common scenarios

For maintainers

• This was checked for breaking API changes and was labeled appropriately

[TinaHeiligers]

@lukeelmers Could you please take a look over the implementation and the tests added to log the ops metrics from the KP?
I'll work on the documentation in the mean time and change the PR to Ready for review if you're happy with it.

[TinaHeiligers]

@elasticmachine merge upstream

[TinaHeiligers]

@elasticmachine merge upstream

[TinaHeiligers]

@elasticmachine merge upstream

[mshustov]

a couple of nits

[TinaHeiligers]

@elasticmachine merge upstream

[TinaHeiligers]

@elasticmachine merge upstream

[kibanamachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request
• Commit: 91eb1ff

Metrics [docs]

✅ unchanged

History

• 💔 Build #101303 failed 2120a31
• 💚 Build #101267 succeeded ed5f6bf
• 💛 Build #101260 was flaky 1714966
• 💚 Build #101224 succeeded 5e7aad7
• 💚 Build #100956 succeeded 64b704e

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream