docs: Restructure and deduplicate security docs (#18594)

libbeat/docs/https.asciidoc

@@ -10,66 +10,156 @@
 //// This content is structured to be included as a whole file.
 //////////////////////////////////////////////////////////////////////////
 
-To secure the communication between {beatname_uc} and Elasticsearch, you can use
-HTTPS and basic authentication. Basic authentication for Elasticsearch is
-available when you enable {security} (see
-{ref}/secure-cluster.html[Secure a cluster] and <<securing-beats>>).
-If you aren't using {security}, you can use a web proxy instead.
+[role="xpack"]
+[[securing-communication-elasticsearch]]
+== Secure communication with Elasticsearch
 
-Here is a sample configuration:
+When sending data to a secured cluster through the `elasticsearch`
+output, {beatname_uc} can use any of the following authentication methods:
 
+* Basic authentication credentials (username and password).
+* Token-based API authentication.
+* A client certificate.
+
+Authentication is specified in the {beatname_uc} configuration file:
+
+* To use *basic authentication*, specify the `username` and `password` settings under `output.elasticsearch`.
+For example:
++
+--
 ["source","yaml",subs="attributes,callouts"]
 ----------------------------------------------------------------------
 output.elasticsearch:
-  username: {beatname_lc} <1>
-  password: verysecret <2>
-  protocol: https <3>
-  hosts: ["elasticsearch.example.com:9200"] <4>
+  hosts: ["https://myEShost:9200"]
+  username: "{beat_default_index_prefix}_writer" <1>
+  password: "{pwd}" <2>
 ----------------------------------------------------------------------
-<1> The username to use for authenticating to Elasticsearch.
-<2> The password to use for authenticating to Elasticsearch.
-<3> This setting enables the HTTPS protocol.
-<4> The IP and port of the Elasticsearch nodes.
-
-TIP: To obfuscate passwords and other sensitive settings,
+<1> This user needs the privileges required to publish events to {es}.
+To create a user like this, see <<privileges-to-publish-events>>.
+<2> This example shows a hard-coded password, but you should store sensitive
+values
 ifndef::serverless[]
-use the <<keystore,secrets keystore>>.
+in the <<keystore,secrets keystore>>.
 endif::[]
 ifdef::serverless[]
-use environment variables.
+in environment variables.
 endif::[]
+--
 
-{beatname_uc} verifies the validity of the server certificates and only accepts trusted
-certificates. Creating a correct SSL/TLS infrastructure is outside the scope of
-this document.
-
-By default {beatname_uc} uses the list of trusted certificate authorities from the
-operating system where {beatname_uc} is running. You can configure {beatname_uc} to use a specific list of
-CA certificates instead of the list from the OS. You can also configure it to use client authentication
-by specifying the certificate and key to use when the server requires the {beatname_uc} to authenticate. Here is an example
-configuration:
+* To use token-based *API key authentication*, specify the `api_key` under `output.elasticsearch`.
+For example:
++
+--
+["source","yaml",subs="attributes,callouts"]
+----------------------------------------------------------------------
+output.elasticsearch:
+  hosts: ["https://myEShost:9200"]
+  api_key: "KnR6yE41RrSowb0kQ0HWoA" <1>
+----------------------------------------------------------------------
+<1> This API key must have the privileges required to publish events to {es}.
+To create an API key like this, see <<beats-api-keys>>.
+--
 
+[[beats-tls]]
+* To use *Public Key Infrastructure (PKI) certificates* to authenticate users,
+specify the `certificate` and `key` settings under `output.elasticsearch`.
+For example:
++
+--
+["source","yaml",subs="attributes,callouts"]
+----------------------------------------------------------------------
+output.elasticsearch:
+  hosts: ["https://myEShost:9200"]
+  ssl.certificate: "/etc/pki/client/cert.pem" <1>
+  ssl.key: "/etc/pki/client/cert.key" <2>
+----------------------------------------------------------------------
+<1> The path to the certificate for SSL client authentication
+<2> The client certificate key
+--
++
+These settings assume that the
+distinguished name (DN) in the certificate is mapped to the appropriate roles in
+the `role_mapping.yml` file on each node in the {es} cluster. For more
+information, see {ref}/mapping-roles.html#mapping-roles-file[Using role
+mapping files].
++
+By default, {beatname_uc} uses the list of trusted certificate authorities (CA) from the
+operating system where {beatname_uc} is running. If the certificate authority that signed your node certificates
+is not in the host system's trusted certificate authorities list, you need
+to add the path to the `.pem` file that contains your CA's certificate to the
+{beatname_uc} configuration. This will configure {beatname_uc} to use a specific list of
+CA certificates instead of the default list from the OS.
++
+Here is an example configuration:
++
+--
 ["source","yaml",subs="attributes,callouts"]
 ----------------------------------------------------------------------
 output.elasticsearch:
-  username: {beatname_lc}
-  password: verysecret
-  protocol: https
-  hosts: ["elasticsearch.example.com:9200"]
+  hosts: ["https://myEShost:9200"]
   ssl.certificate_authorities: <1>
     - /etc/pki/my_root_ca.pem
     - /etc/pki/my_other_ca.pem
   ssl.certificate: "/etc/pki/client.pem" <2>
   ssl.key: "/etc/pki/key.pem" <3>
 ----------------------------------------------------------------------
-<1> The list of CA certificates to trust
+<1> Specify the path to the local `.pem` file that contains your Certificate
+Authority's certificate. This is needed if you use your own CA to sign your node certificates.
 <2> The path to the certificate for SSL client authentication
 <3> The client certificate key
-
+--
++
 NOTE: For any given connection, the SSL/TLS certificates must have a subject
 that matches the value specified for `hosts`, or the SSL handshake fails.
 For example, if you specify `hosts: ["foobar:9200"]`, the certificate MUST
 include `foobar` in the subject (`CN=foobar`) or as a subject alternative name
 (SAN). Make sure the hostname resolves to the correct IP address. If no DNS is available, then
 you can associate the IP address with your hostname in `/etc/hosts`
 (on Unix) or `C:\Windows\System32\drivers\etc\hosts` (on Windows).
+
+ifndef::no_dashboards[]
+[role="xpack"]
+[float]
+[[securing-communication-kibana]]
+=== Secure communication with the Kibana endpoint
+
+If you've configured the <<setup-kibana-endpoint,{kib} endpoint>>,
+you can also specify credentials for authenticating with {kib} under `kibana.setup`.
+If no credentials are specified, Kibana will use the configured authentication method
+in the Elasticsearch output.
+
+For example, specify a unique username and password to connect to Kibana like this:
+
+--
+["source","yaml",subs="attributes,callouts"]
+----
+setup.kibana:
+  host: "mykibanahost:5601"
+  username: "{beat_default_index_prefix}_kib_setup" <1>
+  password: "{pwd}" <2>
+----
+<1> This user needs privileges required to set up dashboards. To create a user like this,
+see <<privileges-to-setup-beats>>.
+<2> This example shows a hard-coded password, but you should store sensitive
+values
+ifndef::serverless[]
+in the <<keystore,secrets keystore>>.
+endif::[]
+ifdef::serverless[]
+in environment variables.
+endif::[]
+endif::no_dashboards[]
+--
+
+[role="xpack"]
+[float]
+[[securing-communication-learn-more]]
+=== Learn more about secure communication
+
+More information on sending data to a secured cluster is available in the configuration reference:
+
+* <<elasticsearch-output>>
+* <<configuration-ssl>>
+ifndef::no_dashboards[]
+* <<setup-kibana-endpoint>>
+endif::no_dashboards[]

libbeat/docs/security/api-keys.asciidoc

@@ -1,27 +1,22 @@
 [role="xpack"]
 [[beats-api-keys]]
-=== Grant access using API keys
+== Grant access using API keys
 
 Instead of using usernames and passwords, you can use API keys to grant
 access to {es} resources. You can set API keys to expire at a certain time,
 and you can explicitly invalidate them. Any user with the `manage_api_key`
 or `manage_own_api_key` cluster privilege can create API keys.
 
-See the {es} API key documentation for more information:
-
-* {ref}/security-api-create-api-key.html[Create API key]
-* {ref}/security-api-get-api-key.html[Get API key information]
-* {ref}/security-api-invalidate-api-key.html[Invalidate API key]
-
 {beatname_uc} instances typically send both collected data and monitoring
 information to {es}. If you are sending both to the same cluster, you can use the same
 API key. For different clusters, you need to use an API key per cluster.
 
 NOTE: For security reasons, we recommend using a unique API key per {beatname_uc} instance.
 You can create as many API keys per user as necessary.
 
+[float]
 [[beats-api-key-publish]]
-==== Create an API key for publishing
+=== Create an API key for publishing
 To create an API key to use for writing data to {es}, use the
 {ref}/security-api-create-api-key.html[Create API key API], for example:
 
@@ -67,8 +62,9 @@ output.elasticsearch:
 --------------------
 <1> Format is `id:api_key` (as returned by {ref}/security-api-create-api-key.html[Create API key])
 
+[float]
 [[beats-api-key-monitor]]
-==== Create an API key for monitoring
+=== Create an API key for monitoring
 To create an API key to use for sending monitoring data to {es}, use the
 {ref}/security-api-create-api-key.html[Create API key API], for example:
 
@@ -114,3 +110,12 @@ monitoring.elasticsearch:
 --------------------
 <1> Format is `id:api_key` (as returned by {ref}/security-api-create-api-key.html[Create API key])
 
+[[learn-more-api-keys]]
+[float]
+=== Learn more about API keys
+
+See the {es} API key documentation for more information:
+
+* {ref}/security-api-create-api-key.html[Create API key]
+* {ref}/security-api-get-api-key.html[Get API key information]
+* {ref}/security-api-invalidate-api-key.html[Invalidate API key]