From 734cdf5080f4e3de7ba696be1c48014766ab9c67 Mon Sep 17 00:00:00 2001 From: DeDe Morton Date: Thu, 9 Jan 2020 18:08:01 -0800 Subject: [PATCH] [DOCS] Clarify changes in monitoring settings (#15003) (#15437) * [DOCS] Clarify changes in monitoring settings * Change topic names and order * Add more fixes from the review * Update libbeat/docs/monitoring/monitoring-beats.asciidoc --- .../docs/monitoring/monitoring-beats.asciidoc | 21 +-- ...toring-internal-collection-legacy.asciidoc | 38 +++++ .../monitoring-internal-collection.asciidoc | 11 +- .../monitoring/monitoring-metricbeat.asciidoc | 6 +- .../shared-monitor-config-legacy.asciidoc | 138 ++++++++++++++++++ .../monitoring/shared-monitor-config.asciidoc | 38 ++--- 6 files changed, 218 insertions(+), 34 deletions(-) create mode 100644 libbeat/docs/monitoring/monitoring-internal-collection-legacy.asciidoc create mode 100644 libbeat/docs/monitoring/shared-monitor-config-legacy.asciidoc diff --git a/libbeat/docs/monitoring/monitoring-beats.asciidoc b/libbeat/docs/monitoring/monitoring-beats.asciidoc index fc6f4c8190f..fd6d28d5d2e 100644 --- a/libbeat/docs/monitoring/monitoring-beats.asciidoc +++ b/libbeat/docs/monitoring/monitoring-beats.asciidoc @@ -14,18 +14,19 @@ ifdef::apm-server[] endif::[] To monitor {beatname_uc}, make sure monitoring is enabled on your {es} cluster, -then configure the method used to collect {beatname_uc} metrics. You -ifndef::serverless[] -can use one of following methods: -endif::[] -ifdef::serverless[] -can use the following method: -endif::[] +then configure the method used to collect {beatname_uc} metrics. You can use one +of following methods: -* <> +* <> - Internal +collectors send monitoring data directly to your monitoring cluster. ifndef::serverless[] -* <> +* <> - +{metricbeat} collects monitoring data from your {beatname_uc} instance +and sends it directly to your monitoring cluster. endif::[] +* <> - +Legacy internal collectors send monitoring data to your production cluster. + //Commenting out this link temporarily until the general monitoring docs can be //updated. @@ -39,3 +40,5 @@ include::monitoring-internal-collection.asciidoc[] ifndef::serverless[] include::monitoring-metricbeat.asciidoc[] endif::[] + +include::monitoring-internal-collection-legacy.asciidoc[] diff --git a/libbeat/docs/monitoring/monitoring-internal-collection-legacy.asciidoc b/libbeat/docs/monitoring/monitoring-internal-collection-legacy.asciidoc new file mode 100644 index 00000000000..6efd1764e0f --- /dev/null +++ b/libbeat/docs/monitoring/monitoring-internal-collection-legacy.asciidoc @@ -0,0 +1,38 @@ +////////////////////////////////////////////////////////////////////////// +//// This content is shared by all Elastic Beats. Make sure you keep the +//// descriptions here generic enough to work for all Beats that include +//// this file. When using cross references, make sure that the cross +//// references resolve correctly for any files that include this one. +//// Use the appropriate variables defined in the index.asciidoc file to +//// resolve Beat names: beatname_uc and beatname_lc. +//// Use the following include to pull this content into a doc file: +//// include::../../libbeat/docs/monitoring/monitoring-internal-collection-legacy.asciidoc[] +////////////////////////////////////////////////////////////////////////// + +[role="xpack"] +[[monitoring-internal-collection-legacy]] +== Use legacy internal collection to send monitoring data +++++ +Legacy internal collection (deprecated) +++++ + +deprecated[7.2.0] + +In {beatname_uc} version 7.1 and earlier, you configured internal collectors +that sent monitoring data to the production cluster, which would either index +the data locally, or forward the data to a dedicated monitoring cluster via HTTP +exporters. + +Starting in {beatname_uc} version 7.2, the legacy settings for internal +collection are deprecated and will be removed in version 8.0.0. Instead of +sending monitoring data to your production cluster, it's recommended that you +use the configuration described under +<> to route +monitoring data directly to your monitoring cluster. + +ifndef::serverless[] +Or as an alternative to internal collection, use +<>. +endif::[] + +include::shared-monitor-config-legacy.asciidoc[] diff --git a/libbeat/docs/monitoring/monitoring-internal-collection.asciidoc b/libbeat/docs/monitoring/monitoring-internal-collection.asciidoc index aa01561d35e..bace51ad5ce 100644 --- a/libbeat/docs/monitoring/monitoring-internal-collection.asciidoc +++ b/libbeat/docs/monitoring/monitoring-internal-collection.asciidoc @@ -11,15 +11,18 @@ [role="xpack"] [[monitoring-internal-collection]] -== Collect {beatname_uc} monitoring data with internal collectors +== Use internal collection to send monitoring data ++++ Internal collection ++++ -The following method involves sending the metrics directly to the monitoring -cluster. +Use internal collectors to send {beats} monitoring data directly to your +monitoring cluster. ifndef::serverless[] -For an alternative method, see <>. +Or as an alternative to internal collection, use +<>. The benefit of using internal collection +instead of {metricbeat} is that you have fewer pieces of software to install +and maintain. endif::[] //Commenting out this link temporarily until the general monitoring docs can be diff --git a/libbeat/docs/monitoring/monitoring-metricbeat.asciidoc b/libbeat/docs/monitoring/monitoring-metricbeat.asciidoc index 2afe6f2bcbc..fba08e6790b 100644 --- a/libbeat/docs/monitoring/monitoring-metricbeat.asciidoc +++ b/libbeat/docs/monitoring/monitoring-metricbeat.asciidoc @@ -1,13 +1,15 @@ [role="xpack"] [[monitoring-metricbeat-collection]] -== Collect {beatname_uc} monitoring data with {metricbeat} +== Use {metricbeat} to send monitoring data [subs="attributes"] ++++ {metricbeat} collection ++++ In 7.3 and later, you can use {metricbeat} to collect data about {beatname_uc} -and ship it to the monitoring cluster. +and ship it to the monitoring cluster. The benefit of using {metricbeat} instead +of internal collection is that the monitoring agent remains active even if the +{beatname_uc} instance dies. ifeval::["{beatname_lc}"=="metricbeat"] Because you'll be using {metricbeat} to _monitor_ {beatname_uc}, you'll need to diff --git a/libbeat/docs/monitoring/shared-monitor-config-legacy.asciidoc b/libbeat/docs/monitoring/shared-monitor-config-legacy.asciidoc new file mode 100644 index 00000000000..d00778f8602 --- /dev/null +++ b/libbeat/docs/monitoring/shared-monitor-config-legacy.asciidoc @@ -0,0 +1,138 @@ +////////////////////////////////////////////////////////////////////////// +//// This content is shared by all Elastic Beats. Make sure you keep the +//// descriptions here generic enough to work for all Beats that include +//// this file. When using cross references, make sure that the cross +//// references resolve correctly for any files that include this one. +//// Use the appropriate variables defined in the index.asciidoc file to +//// resolve Beat names: beatname_uc and beatname_lc. +//// Use the following include to pull this content into a doc file: +//// include::../../libbeat/docs/monitoring/shared-monitor-config.asciidoc[] +//// Make sure this content appears below a level 2 heading. +////////////////////////////////////////////////////////////////////////// + +[role="xpack"] +[[configuration-monitor-legacy]] +=== Settings for legacy internal collection + +deprecated::[7.2.0,These settings are deprecated and will be removed in version 8.0.0. Instead of sending monitoring data to your production cluster it's recommended that you use the configuration described under <> to route monitoring data directly to your monitoring cluster.] + +[float] +=== `xpack.monitoring.enabled` deprecated:[7.2] + +The `enabled` config is a boolean setting to enable or disable {monitoring}. +If set to `true`, monitoring is enabled. + +The default value is `false`. + +[float] +=== `xpack.monitoring.elasticsearch` deprecated:[7.2] + +The {es} instances that you want to ship your {beatname_uc} metrics to. This +configuration option contains the following fields: + +[float] +==== `bulk_max_size` + +The maximum number of metrics to bulk in a single {es} bulk API index request. +The default is `50`. For more information, see <>. + +[float] +==== `backoff.init` + +The number of seconds to wait before trying to reconnect to Elasticsearch after +a network error. After waiting `backoff.init` seconds, {beatname_uc} tries to +reconnect. If the attempt fails, the backoff timer is increased exponentially up +to `backoff.max`. After a successful connection, the backoff timer is reset. The +default is 1s. + +[float] +===== `backoff.max` + +The maximum number of seconds to wait before attempting to connect to +Elasticsearch after a network error. The default is 60s. + +[float] +==== `compression_level` + +The gzip compression level. Setting this value to `0` disables compression. The +compression level must be in the range of `1` (best speed) to `9` (best +compression). The default value is `0`. Increasing the compression level +reduces the network usage but increases the CPU usage. + +[float] +==== `headers` + +Custom HTTP headers to add to each request. For more information, see +<>. + +[float] +==== `hosts` + +The list of {es} nodes to connect to. Monitoring metrics are distributed to +these nodes in round-robin order. For more information, see +<>. + +[float] +==== `max_retries` + +The number of times to retry sending the monitoring metrics after a failure. +After the specified number of retries, the metrics are typically dropped. The +default value is `3`. For more information, see <>. + +[float] +==== `parameters` + +Dictionary of HTTP parameters to pass within the url with index operations. + +[float] +==== `password` + +The password that {beatname_uc} uses to authenticate with the {es} instances for +shipping monitoring data. + +[float] +==== `metrics.period` + +The time interval (in seconds) when metrics are sent to the {es} cluster. A new +snapshot of {beatname_uc} metrics is generated and scheduled for publishing each +period. The default value is 10 * time.Second. + +[float] +==== `state.period` + +The time interval (in seconds) when state information are sent to the {es} cluster. A new +snapshot of {beatname_uc} state is generated and scheduled for publishing each +period. The default value is 60 * time.Second. + +[float] +==== `protocol` + +The name of the protocol to use when connecting to the {es} cluster. The options +are: `http` or `https`. The default is `http`. If you specify a URL for `hosts`, +however, the value of protocol is overridden by the scheme you specify in the URL. + +[float] +==== `proxy_url` + +The URL of the proxy to use when connecting to the {es} cluster. For more +information, see <>. + +[float] +==== `timeout` + +The HTTP request timeout in seconds for the {es} request. The default is `90`. + +[float] +==== `ssl` + +Configuration options for Transport Layer Security (TLS) or Secure Sockets Layer +(SSL) parameters like the certificate authority (CA) to use for HTTPS-based +connections. If the `ssl` section is missing, the host CAs are used for +HTTPS connections to {es}. For more information, see <>. + +[float] +==== `username` + +The user ID that {beatname_uc} uses to authenticate with the {es} instances for +shipping monitoring data. + diff --git a/libbeat/docs/monitoring/shared-monitor-config.asciidoc b/libbeat/docs/monitoring/shared-monitor-config.asciidoc index 1b249078f25..609c3c3eb09 100644 --- a/libbeat/docs/monitoring/shared-monitor-config.asciidoc +++ b/libbeat/docs/monitoring/shared-monitor-config.asciidoc @@ -6,13 +6,13 @@ //// Use the appropriate variables defined in the index.asciidoc file to //// resolve Beat names: beatname_uc and beatname_lc. //// Use the following include to pull this content into a doc file: -//// include::../../libbeat/docs/monitoring/configuring.asciidoc[] +//// include::../../libbeat/docs/monitoring/shared-monitor-config.asciidoc[] //// Make sure this content appears below a level 2 heading. ////////////////////////////////////////////////////////////////////////// [role="xpack"] [[configuration-monitor]] -=== Settings for internal monitoring collection +=== Settings for internal collection Use the following settings to configure internal collection when you are not using {metricbeat} to collect monitoring data. @@ -32,17 +32,17 @@ The default value is `false`. The {es} instances that you want to ship your {beatname_uc} metrics to. This configuration option contains the following fields: -==== `api_key` +===== `api_key` The detail of the API key to be used to send monitoring information to {es}. See <> for more information. -==== `bulk_max_size` +===== `bulk_max_size` The maximum number of metrics to bulk in a single {es} bulk API index request. The default is `50`. For more information, see <>. -==== `backoff.init` +===== `backoff.init` The number of seconds to wait before trying to reconnect to Elasticsearch after a network error. After waiting `backoff.init` seconds, {beatname_uc} tries to @@ -50,79 +50,79 @@ reconnect. If the attempt fails, the backoff timer is increased exponentially up to `backoff.max`. After a successful connection, the backoff timer is reset. The default is 1s. -==== `backoff.max` +===== `backoff.max` The maximum number of seconds to wait before attempting to connect to Elasticsearch after a network error. The default is 60s. -==== `compression_level` +===== `compression_level` The gzip compression level. Setting this value to `0` disables compression. The compression level must be in the range of `1` (best speed) to `9` (best compression). The default value is `0`. Increasing the compression level reduces the network usage but increases the CPU usage. -==== `headers` +===== `headers` Custom HTTP headers to add to each request. For more information, see <>. -==== `hosts` +===== `hosts` The list of {es} nodes to connect to. Monitoring metrics are distributed to these nodes in round robin order. For more information, see <>. -==== `max_retries` +===== `max_retries` The number of times to retry sending the monitoring metrics after a failure. After the specified number of retries, the metrics are typically dropped. The default value is `3`. For more information, see <>. -==== `parameters` +===== `parameters` Dictionary of HTTP parameters to pass within the url with index operations. -==== `password` +===== `password` The password that {beatname_uc} uses to authenticate with the {es} instances for shipping monitoring data. -==== `metrics.period` +===== `metrics.period` The time interval (in seconds) when metrics are sent to the {es} cluster. A new snapshot of {beatname_uc} metrics is generated and scheduled for publishing each period. The default value is 10 * time.Second. -==== `state.period` +===== `state.period` The time interval (in seconds) when state information are sent to the {es} cluster. A new snapshot of {beatname_uc} state is generated and scheduled for publishing each period. The default value is 60 * time.Second. -==== `protocol` +===== `protocol` The name of the protocol to use when connecting to the {es} cluster. The options are: `http` or `https`. The default is `http`. If you specify a URL for `hosts`, however, the value of protocol is overridden by the scheme you specify in the URL. -==== `proxy_url` +===== `proxy_url` The URL of the proxy to use when connecting to the {es} cluster. For more information, see <>. -==== `timeout` +===== `timeout` The HTTP request timeout in seconds for the {es} request. The default is `90`. -==== `ssl` +===== `ssl` Configuration options for Transport Layer Security (TLS) or Secure Sockets Layer (SSL) parameters like the certificate authority (CA) to use for HTTPS-based connections. If the `ssl` section is missing, the host CAs are used for HTTPS connections to {es}. For more information, see <>. -==== `username` +===== `username` The user ID that {beatname_uc} uses to authenticate with the {es} instances for shipping monitoring data.