From 291d76bc461158812fb7487badfcf6e7f75e55fb Mon Sep 17 00:00:00 2001 From: DeDe Morton Date: Thu, 9 Jan 2020 18:10:09 -0800 Subject: [PATCH] [7.3][DOCS] Backport: Clarify changes in monitoring settings (#15003) (#15440) * [DOCS] Clarify changes in monitoring settings (#15003) * [DOCS] Clarify changes in monitoring settings * Change topic names and order * Add more fixes from the review * Update libbeat/docs/monitoring/monitoring-beats.asciidoc * Update the notice file --- NOTICE.txt | 2 +- .../docs/monitoring/monitoring-beats.asciidoc | 21 +-- ...toring-internal-collection-legacy.asciidoc | 38 +++++ .../monitoring-internal-collection.asciidoc | 11 +- .../monitoring/monitoring-metricbeat.asciidoc | 7 +- .../shared-monitor-config-legacy.asciidoc | 138 ++++++++++++++++++ .../monitoring/shared-monitor-config.asciidoc | 36 ++--- 7 files changed, 218 insertions(+), 35 deletions(-) create mode 100644 libbeat/docs/monitoring/monitoring-internal-collection-legacy.asciidoc create mode 100644 libbeat/docs/monitoring/shared-monitor-config-legacy.asciidoc diff --git a/NOTICE.txt b/NOTICE.txt index e77694d2ef2..6f25e544eda 100644 --- a/NOTICE.txt +++ b/NOTICE.txt @@ -1,5 +1,5 @@ Elastic Beats -Copyright 2014-2019 Elasticsearch BV +Copyright 2014-2020 Elasticsearch BV This product includes software developed by The Apache Software Foundation (http://www.apache.org/). diff --git a/libbeat/docs/monitoring/monitoring-beats.asciidoc b/libbeat/docs/monitoring/monitoring-beats.asciidoc index d13842aa072..78c0021d841 100644 --- a/libbeat/docs/monitoring/monitoring-beats.asciidoc +++ b/libbeat/docs/monitoring/monitoring-beats.asciidoc @@ -9,18 +9,19 @@ You can use the {stack} {monitor-features} to gain insight into the health of {beatname_uc} agents running in your environment. To monitor {beatname_uc}, make sure monitoring is enabled on your {es} cluster, -then configure the method used to collect {beatname_uc} metrics. You -ifndef::serverless[] -can use one of following methods: -endif::[] -ifdef::serverless[] -can use the following method: -endif::[] +then configure the method used to collect {beatname_uc} metrics. You can use one +of following methods: -* <> +* <> - Internal +collectors send monitoring data directly to your monitoring cluster. ifndef::serverless[] -* <> +* <> - +{metricbeat} collects monitoring data from your {beatname_uc} instance +and sends it directly to your monitoring cluster. endif::[] +* <> - +Legacy internal collectors send monitoring data to your production cluster. + To learn about monitoring in general, see {stack-ov}/xpack-monitoring.html[Monitoring the {stack}]. @@ -32,3 +33,5 @@ include::monitoring-internal-collection.asciidoc[] ifndef::serverless[] include::monitoring-metricbeat.asciidoc[] endif::[] + +include::monitoring-internal-collection-legacy.asciidoc[] diff --git a/libbeat/docs/monitoring/monitoring-internal-collection-legacy.asciidoc b/libbeat/docs/monitoring/monitoring-internal-collection-legacy.asciidoc new file mode 100644 index 00000000000..6efd1764e0f --- /dev/null +++ b/libbeat/docs/monitoring/monitoring-internal-collection-legacy.asciidoc @@ -0,0 +1,38 @@ +////////////////////////////////////////////////////////////////////////// +//// This content is shared by all Elastic Beats. Make sure you keep the +//// descriptions here generic enough to work for all Beats that include +//// this file. When using cross references, make sure that the cross +//// references resolve correctly for any files that include this one. +//// Use the appropriate variables defined in the index.asciidoc file to +//// resolve Beat names: beatname_uc and beatname_lc. +//// Use the following include to pull this content into a doc file: +//// include::../../libbeat/docs/monitoring/monitoring-internal-collection-legacy.asciidoc[] +////////////////////////////////////////////////////////////////////////// + +[role="xpack"] +[[monitoring-internal-collection-legacy]] +== Use legacy internal collection to send monitoring data +++++ +Legacy internal collection (deprecated) +++++ + +deprecated[7.2.0] + +In {beatname_uc} version 7.1 and earlier, you configured internal collectors +that sent monitoring data to the production cluster, which would either index +the data locally, or forward the data to a dedicated monitoring cluster via HTTP +exporters. + +Starting in {beatname_uc} version 7.2, the legacy settings for internal +collection are deprecated and will be removed in version 8.0.0. Instead of +sending monitoring data to your production cluster, it's recommended that you +use the configuration described under +<> to route +monitoring data directly to your monitoring cluster. + +ifndef::serverless[] +Or as an alternative to internal collection, use +<>. +endif::[] + +include::shared-monitor-config-legacy.asciidoc[] diff --git a/libbeat/docs/monitoring/monitoring-internal-collection.asciidoc b/libbeat/docs/monitoring/monitoring-internal-collection.asciidoc index 3ea3c47e158..0b8b0b81734 100644 --- a/libbeat/docs/monitoring/monitoring-internal-collection.asciidoc +++ b/libbeat/docs/monitoring/monitoring-internal-collection.asciidoc @@ -11,15 +11,18 @@ [role="xpack"] [[monitoring-internal-collection]] -== Collect {beatname_uc} monitoring data with internal collectors +== Use internal collection to send monitoring data ++++ Internal collection ++++ -The following method involves sending the metrics to the production cluster, -which ultimately routes them to the monitoring cluster. +Use internal collectors to send {beats} monitoring data directly to your +monitoring cluster. ifndef::serverless[] -For an alternative method, see <>. +Or as an alternative to internal collection, use +<>. The benefit of using internal collection +instead of {metricbeat} is that you have fewer pieces of software to install +and maintain. endif::[] To learn about monitoring in general, see diff --git a/libbeat/docs/monitoring/monitoring-metricbeat.asciidoc b/libbeat/docs/monitoring/monitoring-metricbeat.asciidoc index 7afdba800c2..8c958ff557f 100644 --- a/libbeat/docs/monitoring/monitoring-metricbeat.asciidoc +++ b/libbeat/docs/monitoring/monitoring-metricbeat.asciidoc @@ -1,14 +1,15 @@ [role="xpack"] [[monitoring-metricbeat-collection]] -== Collect {beatname_uc} monitoring data with {metricbeat} +== Use {metricbeat} to send monitoring data [subs="attributes"] ++++ {metricbeat} collection ++++ In 7.3 and later, you can use {metricbeat} to collect data about {beatname_uc} -and ship it to the monitoring cluster, rather than routing it through the -production cluster as described in <>. +and ship it to the monitoring cluster. The benefit of using {metricbeat} instead +of internal collection is that the monitoring agent remains active even if the +{beatname_uc} instance dies. ifeval::["{beatname_lc}"=="metricbeat"] Because you'll be using {metricbeat} to _monitor_ {beatname_uc}, you'll need to diff --git a/libbeat/docs/monitoring/shared-monitor-config-legacy.asciidoc b/libbeat/docs/monitoring/shared-monitor-config-legacy.asciidoc new file mode 100644 index 00000000000..d00778f8602 --- /dev/null +++ b/libbeat/docs/monitoring/shared-monitor-config-legacy.asciidoc @@ -0,0 +1,138 @@ +////////////////////////////////////////////////////////////////////////// +//// This content is shared by all Elastic Beats. Make sure you keep the +//// descriptions here generic enough to work for all Beats that include +//// this file. When using cross references, make sure that the cross +//// references resolve correctly for any files that include this one. +//// Use the appropriate variables defined in the index.asciidoc file to +//// resolve Beat names: beatname_uc and beatname_lc. +//// Use the following include to pull this content into a doc file: +//// include::../../libbeat/docs/monitoring/shared-monitor-config.asciidoc[] +//// Make sure this content appears below a level 2 heading. +////////////////////////////////////////////////////////////////////////// + +[role="xpack"] +[[configuration-monitor-legacy]] +=== Settings for legacy internal collection + +deprecated::[7.2.0,These settings are deprecated and will be removed in version 8.0.0. Instead of sending monitoring data to your production cluster it's recommended that you use the configuration described under <> to route monitoring data directly to your monitoring cluster.] + +[float] +=== `xpack.monitoring.enabled` deprecated:[7.2] + +The `enabled` config is a boolean setting to enable or disable {monitoring}. +If set to `true`, monitoring is enabled. + +The default value is `false`. + +[float] +=== `xpack.monitoring.elasticsearch` deprecated:[7.2] + +The {es} instances that you want to ship your {beatname_uc} metrics to. This +configuration option contains the following fields: + +[float] +==== `bulk_max_size` + +The maximum number of metrics to bulk in a single {es} bulk API index request. +The default is `50`. For more information, see <>. + +[float] +==== `backoff.init` + +The number of seconds to wait before trying to reconnect to Elasticsearch after +a network error. After waiting `backoff.init` seconds, {beatname_uc} tries to +reconnect. If the attempt fails, the backoff timer is increased exponentially up +to `backoff.max`. After a successful connection, the backoff timer is reset. The +default is 1s. + +[float] +===== `backoff.max` + +The maximum number of seconds to wait before attempting to connect to +Elasticsearch after a network error. The default is 60s. + +[float] +==== `compression_level` + +The gzip compression level. Setting this value to `0` disables compression. The +compression level must be in the range of `1` (best speed) to `9` (best +compression). The default value is `0`. Increasing the compression level +reduces the network usage but increases the CPU usage. + +[float] +==== `headers` + +Custom HTTP headers to add to each request. For more information, see +<>. + +[float] +==== `hosts` + +The list of {es} nodes to connect to. Monitoring metrics are distributed to +these nodes in round-robin order. For more information, see +<>. + +[float] +==== `max_retries` + +The number of times to retry sending the monitoring metrics after a failure. +After the specified number of retries, the metrics are typically dropped. The +default value is `3`. For more information, see <>. + +[float] +==== `parameters` + +Dictionary of HTTP parameters to pass within the url with index operations. + +[float] +==== `password` + +The password that {beatname_uc} uses to authenticate with the {es} instances for +shipping monitoring data. + +[float] +==== `metrics.period` + +The time interval (in seconds) when metrics are sent to the {es} cluster. A new +snapshot of {beatname_uc} metrics is generated and scheduled for publishing each +period. The default value is 10 * time.Second. + +[float] +==== `state.period` + +The time interval (in seconds) when state information are sent to the {es} cluster. A new +snapshot of {beatname_uc} state is generated and scheduled for publishing each +period. The default value is 60 * time.Second. + +[float] +==== `protocol` + +The name of the protocol to use when connecting to the {es} cluster. The options +are: `http` or `https`. The default is `http`. If you specify a URL for `hosts`, +however, the value of protocol is overridden by the scheme you specify in the URL. + +[float] +==== `proxy_url` + +The URL of the proxy to use when connecting to the {es} cluster. For more +information, see <>. + +[float] +==== `timeout` + +The HTTP request timeout in seconds for the {es} request. The default is `90`. + +[float] +==== `ssl` + +Configuration options for Transport Layer Security (TLS) or Secure Sockets Layer +(SSL) parameters like the certificate authority (CA) to use for HTTPS-based +connections. If the `ssl` section is missing, the host CAs are used for +HTTPS connections to {es}. For more information, see <>. + +[float] +==== `username` + +The user ID that {beatname_uc} uses to authenticate with the {es} instances for +shipping monitoring data. + diff --git a/libbeat/docs/monitoring/shared-monitor-config.asciidoc b/libbeat/docs/monitoring/shared-monitor-config.asciidoc index f7dabd8df4e..20a498ea78d 100644 --- a/libbeat/docs/monitoring/shared-monitor-config.asciidoc +++ b/libbeat/docs/monitoring/shared-monitor-config.asciidoc @@ -6,13 +6,13 @@ //// Use the appropriate variables defined in the index.asciidoc file to //// resolve Beat names: beatname_uc and beatname_lc. //// Use the following include to pull this content into a doc file: -//// include::../../libbeat/docs/monitoring/configuring.asciidoc[] +//// include::../../libbeat/docs/monitoring/shared-monitor-config.asciidoc[] //// Make sure this content appears below a level 2 heading. ////////////////////////////////////////////////////////////////////////// [role="xpack"] [[configuration-monitor]] -=== Settings for internal monitoring collection +=== Settings for internal collection Use the following settings to configure internal collection when you are not using {metricbeat} to collect monitoring data. @@ -32,12 +32,12 @@ The default value is `false`. The {es} instances that you want to ship your {beatname_uc} metrics to. This configuration option contains the following fields: -==== `bulk_max_size` +===== `bulk_max_size` The maximum number of metrics to bulk in a single {es} bulk API index request. The default is `50`. For more information, see <>. -==== `backoff.init` +===== `backoff.init` The number of seconds to wait before trying to reconnect to Elasticsearch after a network error. After waiting `backoff.init` seconds, {beatname_uc} tries to @@ -45,79 +45,79 @@ reconnect. If the attempt fails, the backoff timer is increased exponentially up to `backoff.max`. After a successful connection, the backoff timer is reset. The default is 1s. -==== `backoff.max` +===== `backoff.max` The maximum number of seconds to wait before attempting to connect to Elasticsearch after a network error. The default is 60s. -==== `compression_level` +===== `compression_level` The gzip compression level. Setting this value to `0` disables compression. The compression level must be in the range of `1` (best speed) to `9` (best compression). The default value is `0`. Increasing the compression level reduces the network usage but increases the CPU usage. -==== `headers` +===== `headers` Custom HTTP headers to add to each request. For more information, see <>. -==== `hosts` +===== `hosts` The list of {es} nodes to connect to. Monitoring metrics are distributed to these nodes in round robin order. For more information, see <>. -==== `max_retries` +===== `max_retries` The number of times to retry sending the monitoring metrics after a failure. After the specified number of retries, the metrics are typically dropped. The default value is `3`. For more information, see <>. -==== `parameters` +===== `parameters` Dictionary of HTTP parameters to pass within the url with index operations. -==== `password` +===== `password` The password that {beatname_uc} uses to authenticate with the {es} instances for shipping monitoring data. -==== `metrics.period` +===== `metrics.period` The time interval (in seconds) when metrics are sent to the {es} cluster. A new snapshot of {beatname_uc} metrics is generated and scheduled for publishing each period. The default value is 10 * time.Second. -==== `state.period` +===== `state.period` The time interval (in seconds) when state information are sent to the {es} cluster. A new snapshot of {beatname_uc} state is generated and scheduled for publishing each period. The default value is 60 * time.Second. -==== `protocol` +===== `protocol` The name of the protocol to use when connecting to the {es} cluster. The options are: `http` or `https`. The default is `http`. If you specify a URL for `hosts`, however, the value of protocol is overridden by the scheme you specify in the URL. -==== `proxy_url` +===== `proxy_url` The URL of the proxy to use when connecting to the {es} cluster. For more information, see <>. -==== `timeout` +===== `timeout` The HTTP request timeout in seconds for the {es} request. The default is `90`. -==== `ssl` +===== `ssl` Configuration options for Transport Layer Security (TLS) or Secure Sockets Layer (SSL) parameters like the certificate authority (CA) to use for HTTPS-based connections. If the `ssl` section is missing, the host CAs are used for HTTPS connections to {es}. For more information, see <>. -==== `username` +===== `username` The user ID that {beatname_uc} uses to authenticate with the {es} instances for shipping monitoring data.