[7.3][DOCS] Backport: Clarify changes in monitoring settings (#15003) (…

…#15440)

* [DOCS] Clarify changes in monitoring settings (#15003)

* [DOCS] Clarify changes in monitoring settings

* Change topic names and order

* Add more fixes from the review

* Update libbeat/docs/monitoring/monitoring-beats.asciidoc

* Update the notice file

NOTICE.txt

@@ -1,5 +1,5 @@
 Elastic Beats
-Copyright 2014-2019 Elasticsearch BV
+Copyright 2014-2020 Elasticsearch BV
 
 This product includes software developed by The Apache Software 
 Foundation (http://www.apache.org/).

libbeat/docs/monitoring/monitoring-beats.asciidoc

@@ -9,18 +9,19 @@ You can use the {stack} {monitor-features} to gain insight into the health of
 {beatname_uc} agents running in your environment. 
 
 To monitor {beatname_uc}, make sure monitoring is enabled on your {es} cluster,
-then configure the method used to collect {beatname_uc} metrics. You
-ifndef::serverless[]
-can use one of following methods:
-endif::[]
-ifdef::serverless[]
-can use the following method:
-endif::[]
+then configure the method used to collect {beatname_uc} metrics. You can use one
+of following methods:
 
-* <<monitoring-internal-collection,Internal collection>>
+* <<monitoring-internal-collection,Internal collection>> - Internal
+collectors send monitoring data directly to your monitoring cluster.
 ifndef::serverless[]
-* <<monitoring-metricbeat-collection, {metricbeat} collection>>
+* <<monitoring-metricbeat-collection, {metricbeat} collection>> - 
+{metricbeat} collects monitoring data from your {beatname_uc} instance
+and sends it directly to your monitoring cluster.
 endif::[]
+* <<monitoring-internal-collection-legacy,Legacy internal collection (deprecated)>> - 
+Legacy internal collectors send monitoring data to your production cluster.
+
 
 To learn about monitoring in general, see 
 {stack-ov}/xpack-monitoring.html[Monitoring the {stack}]. 
@@ -32,3 +33,5 @@ include::monitoring-internal-collection.asciidoc[]
 ifndef::serverless[]
 include::monitoring-metricbeat.asciidoc[]
 endif::[]
+
+include::monitoring-internal-collection-legacy.asciidoc[]

libbeat/docs/monitoring/monitoring-internal-collection-legacy.asciidoc

@@ -0,0 +1,38 @@
+//////////////////////////////////////////////////////////////////////////
+//// This content is shared by all Elastic Beats. Make sure you keep the
+//// descriptions here generic enough to work for all Beats that include
+//// this file. When using cross references, make sure that the cross
+//// references resolve correctly for any files that include this one.
+//// Use the appropriate variables defined in the index.asciidoc file to
+//// resolve Beat names: beatname_uc and beatname_lc.
+//// Use the following include to pull this content into a doc file:
+//// include::../../libbeat/docs/monitoring/monitoring-internal-collection-legacy.asciidoc[]
+//////////////////////////////////////////////////////////////////////////
+
+[role="xpack"]
+[[monitoring-internal-collection-legacy]]
+== Use legacy internal collection to send monitoring data
+++++
+<titleabbrev>Legacy internal collection (deprecated)</titleabbrev>
+++++
+
+deprecated[7.2.0]
+
+In {beatname_uc} version 7.1 and earlier, you configured internal collectors
+that sent monitoring data to the production cluster, which would either index
+the data locally, or forward the data to a dedicated monitoring cluster via HTTP
+exporters.
+
+Starting in {beatname_uc} version 7.2, the legacy settings for internal
+collection are deprecated and will be removed in version 8.0.0. Instead of
+sending monitoring data to your production cluster, it's recommended that you
+use the configuration described under
+<<monitoring-internal-collection,internal collection>> to route
+monitoring data directly to your monitoring cluster.
+
+ifndef::serverless[]
+Or as an alternative to internal collection, use
+<<monitoring-metricbeat-collection>>.
+endif::[]
+
+include::shared-monitor-config-legacy.asciidoc[]

libbeat/docs/monitoring/monitoring-internal-collection.asciidoc

@@ -11,15 +11,18 @@
 
 [role="xpack"]
 [[monitoring-internal-collection]]
-== Collect {beatname_uc} monitoring data with internal collectors
+== Use internal collection to send monitoring data
 ++++
 <titleabbrev>Internal collection</titleabbrev>
 ++++
 
-The following method involves sending the metrics to the production cluster, 
-which ultimately routes them to the monitoring cluster.
+Use internal collectors to send {beats} monitoring data directly to your
+monitoring cluster.
 ifndef::serverless[]
-For an alternative method, see <<monitoring-metricbeat-collection>>.
+Or as an alternative to internal collection, use
+<<monitoring-metricbeat-collection>>. The benefit of using internal collection
+instead of {metricbeat} is that you have fewer pieces of software to install
+and maintain.
 endif::[]
 
 To learn about monitoring in general, see 

libbeat/docs/monitoring/monitoring-metricbeat.asciidoc

@@ -1,14 +1,15 @@
 [role="xpack"]
 [[monitoring-metricbeat-collection]]
-== Collect {beatname_uc} monitoring data with {metricbeat}
+== Use {metricbeat} to send monitoring data
 [subs="attributes"]
 ++++
 <titleabbrev>{metricbeat} collection</titleabbrev>
 ++++
 
 In 7.3 and later, you can use {metricbeat} to collect data about {beatname_uc} 
-and ship it to the monitoring cluster, rather than routing it through the 
-production cluster as described in <<monitoring-internal-collection>>.
+and ship it to the monitoring cluster. The benefit of using {metricbeat} instead
+of internal collection is that the monitoring agent remains active even if the
+{beatname_uc} instance dies.
 
 ifeval::["{beatname_lc}"=="metricbeat"]
 Because you'll be using {metricbeat} to _monitor_ {beatname_uc}, you'll need to

libbeat/docs/monitoring/shared-monitor-config-legacy.asciidoc

@@ -0,0 +1,138 @@
+//////////////////////////////////////////////////////////////////////////
+//// This content is shared by all Elastic Beats. Make sure you keep the
+//// descriptions here generic enough to work for all Beats that include
+//// this file. When using cross references, make sure that the cross
+//// references resolve correctly for any files that include this one.
+//// Use the appropriate variables defined in the index.asciidoc file to
+//// resolve Beat names: beatname_uc and beatname_lc.
+//// Use the following include to pull this content into a doc file:
+//// include::../../libbeat/docs/monitoring/shared-monitor-config.asciidoc[]
+//// Make sure this content appears below a level 2 heading.
+//////////////////////////////////////////////////////////////////////////
+
+[role="xpack"]
+[[configuration-monitor-legacy]]
+=== Settings for legacy internal collection
+
+deprecated::[7.2.0,These settings are deprecated and will be removed in version 8.0.0. Instead of sending monitoring data to your production cluster it's recommended that you use the configuration described under <<monitoring-internal-collection>> to route monitoring data directly to your monitoring cluster.]
+
+[float]
+=== `xpack.monitoring.enabled` deprecated:[7.2]
+
+The `enabled` config is a boolean setting to enable or disable {monitoring}.
+If set to `true`, monitoring is enabled.
+
+The default value is `false`.
+
+[float]
+=== `xpack.monitoring.elasticsearch` deprecated:[7.2]
+
+The {es} instances that you want to ship your {beatname_uc} metrics to. This
+configuration option contains the following fields:
+
+[float]
+==== `bulk_max_size`
+
+The maximum number of metrics to bulk in a single {es} bulk API index request.
+The default is `50`. For more information, see <<elasticsearch-output>>.
+
+[float]
+==== `backoff.init`
+
+The number of seconds to wait before trying to reconnect to Elasticsearch after
+a network error. After waiting `backoff.init` seconds, {beatname_uc} tries to
+reconnect. If the attempt fails, the backoff timer is increased exponentially up
+to `backoff.max`. After a successful connection, the backoff timer is reset. The
+default is 1s.
+
+[float]
+===== `backoff.max`
+
+The maximum number of seconds to wait before attempting to connect to
+Elasticsearch after a network error. The default is 60s.
+
+[float]
+==== `compression_level`
+
+The gzip compression level. Setting this value to `0` disables compression. The
+compression level must be in the range of `1` (best speed) to `9` (best
+compression). The default value is `0`. Increasing the compression level
+reduces the network usage but increases the CPU usage.
+
+[float]
+==== `headers`
+
+Custom HTTP headers to add to each request. For more information, see
+<<elasticsearch-output>>.
+
+[float]
+==== `hosts`
+
+The list of {es} nodes to connect to. Monitoring metrics are distributed to
+these nodes in round-robin order. For more information, see
+<<elasticsearch-output>>.
+
+[float]
+==== `max_retries`
+
+The number of times to retry sending the monitoring metrics after a failure.
+After the specified number of retries, the metrics are typically dropped. The
+default value is `3`. For more information, see <<elasticsearch-output>>.
+
+[float]
+==== `parameters`
+
+Dictionary of HTTP parameters to pass within the url with index operations.
+
+[float]
+==== `password`
+
+The password that {beatname_uc} uses to authenticate with the {es} instances for
+shipping monitoring data.
+
+[float]
+==== `metrics.period`
+
+The time interval (in seconds) when metrics are sent to the {es} cluster. A new
+snapshot of {beatname_uc} metrics is generated and scheduled for publishing each
+period. The default value is 10 * time.Second.
+
+[float]
+==== `state.period`
+
+The time interval (in seconds) when state information are sent to the {es} cluster. A new
+snapshot of {beatname_uc} state is generated and scheduled for publishing each
+period. The default value is 60 * time.Second.
+
+[float]
+==== `protocol`
+
+The name of the protocol to use when connecting to the {es} cluster. The options
+are: `http` or `https`. The default is `http`. If you specify a URL for `hosts`,
+however, the value of protocol is overridden by the scheme you specify in the URL.
+
+[float]
+==== `proxy_url`
+
+The URL of the proxy to use when connecting to the {es} cluster. For more
+information, see <<elasticsearch-output>>.
+
+[float]
+==== `timeout`
+
+The HTTP request timeout in seconds for the {es} request. The default is `90`.
+
+[float]
+==== `ssl`
+
+Configuration options for Transport Layer Security (TLS) or Secure Sockets Layer
+(SSL) parameters like the certificate authority (CA) to use for HTTPS-based
+connections. If the `ssl` section is missing, the host CAs are used for
+HTTPS connections to {es}. For more information, see <<configuration-ssl>>.
+
+[float]
+==== `username`
+
+The user ID that {beatname_uc} uses to authenticate with the {es} instances for
+shipping monitoring data.
+

libbeat/docs/monitoring/shared-monitor-config.asciidoc

@@ -6,13 +6,13 @@
 //// Use the appropriate variables defined in the index.asciidoc file to
 //// resolve Beat names: beatname_uc and beatname_lc.
 //// Use the following include to pull this content into a doc file:
-//// include::../../libbeat/docs/monitoring/configuring.asciidoc[]
+//// include::../../libbeat/docs/monitoring/shared-monitor-config.asciidoc[]
 //// Make sure this content appears below a level 2 heading.
 //////////////////////////////////////////////////////////////////////////
 
 [role="xpack"]
 [[configuration-monitor]]
-=== Settings for internal monitoring collection
+=== Settings for internal collection
 
 Use the following settings to configure internal collection when you are not
 using {metricbeat} to collect monitoring data.
@@ -32,92 +32,92 @@ The default value is `false`.
 The {es} instances that you want to ship your {beatname_uc} metrics to. This
 configuration option contains the following fields:
 
-==== `bulk_max_size`
+===== `bulk_max_size`
 
 The maximum number of metrics to bulk in a single {es} bulk API index request.
 The default is `50`. For more information, see <<elasticsearch-output>>.
 
-==== `backoff.init`
+===== `backoff.init`
 
 The number of seconds to wait before trying to reconnect to Elasticsearch after
 a network error. After waiting `backoff.init` seconds, {beatname_uc} tries to
 reconnect. If the attempt fails, the backoff timer is increased exponentially up
 to `backoff.max`. After a successful connection, the backoff timer is reset. The
 default is 1s.
 
-==== `backoff.max`
+===== `backoff.max`
 
 The maximum number of seconds to wait before attempting to connect to
 Elasticsearch after a network error. The default is 60s.
 
-==== `compression_level`
+===== `compression_level`
 
 The gzip compression level. Setting this value to `0` disables compression. The
 compression level must be in the range of `1` (best speed) to `9` (best
 compression). The default value is `0`. Increasing the compression level
 reduces the network usage but increases the CPU usage.
 
-==== `headers`
+===== `headers`
 
 Custom HTTP headers to add to each request. For more information, see
 <<elasticsearch-output>>.
 
-==== `hosts`
+===== `hosts`
 
 The list of {es} nodes to connect to. Monitoring metrics are distributed to
 these nodes in round robin order. For more information, see
 <<elasticsearch-output>>.
 
-==== `max_retries`
+===== `max_retries`
 
 The number of times to retry sending the monitoring metrics after a failure.
 After the specified number of retries, the metrics are typically dropped. The
 default value is `3`. For more information, see <<elasticsearch-output>>.
 
-==== `parameters`
+===== `parameters`
 
 Dictionary of HTTP parameters to pass within the url with index operations.
 
-==== `password`
+===== `password`
 
 The password that {beatname_uc} uses to authenticate with the {es} instances for
 shipping monitoring data.
 
-==== `metrics.period`
+===== `metrics.period`
 
 The time interval (in seconds) when metrics are sent to the {es} cluster. A new
 snapshot of {beatname_uc} metrics is generated and scheduled for publishing each
 period. The default value is 10 * time.Second.
 
-==== `state.period`
+===== `state.period`
 
 The time interval (in seconds) when state information are sent to the {es} cluster. A new
 snapshot of {beatname_uc} state is generated and scheduled for publishing each
 period. The default value is 60 * time.Second.
 
-==== `protocol`
+===== `protocol`
 
 The name of the protocol to use when connecting to the {es} cluster. The options
 are: `http` or `https`. The default is `http`. If you specify a URL for `hosts`,
 however, the value of protocol is overridden by the scheme you specify in the URL.
 
-==== `proxy_url`
+===== `proxy_url`
 
 The URL of the proxy to use when connecting to the {es} cluster. For more
 information, see <<elasticsearch-output>>.
 
-==== `timeout`
+===== `timeout`
 
 The HTTP request timeout in seconds for the {es} request. The default is `90`.
 
-==== `ssl`
+===== `ssl`
 
 Configuration options for Transport Layer Security (TLS) or Secure Sockets Layer
 (SSL) parameters like the certificate authority (CA) to use for HTTPS-based
 connections. If the `ssl` section is missing, the host CAs are used for
 HTTPS connections to {es}. For more information, see <<configuration-ssl>>.
 
-==== `username`
+===== `username`
 
 The user ID that {beatname_uc} uses to authenticate with the {es} instances for
 shipping monitoring data.