-
Notifications
You must be signed in to change notification settings - Fork 161
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add proxy setup docs for Elastic Agent and Fleet (#1239)
* Add proxy setup docs for Elastic Agent and Fleet * Add changes from review * More changes from review * Apply additional chagnes from the review * Add section about where to set env vars * Add revisions from review * Resolve reamining commments from mostlyjason
- Loading branch information
1 parent
96f38ce
commit 7e4e5d6
Showing
6 changed files
with
319 additions
and
0 deletions.
There are no files selected for viewing
317 changes: 317 additions & 0 deletions
317
docs/en/ingest-management/fleet-agent-proxy-support.asciidoc
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,317 @@ | ||
[[fleet-agent-proxy-support]] | ||
= Use a proxy server with {agent} and {fleet} | ||
|
||
Your organization’s security strategy and other considerations may require you | ||
to use a proxy server between some components in your deployment. For example, | ||
you may have a firewall rule that prevents endpoints from connecting directly to | ||
{es}. In this scenario, you can set up the {agent} to connect to a proxy, then | ||
the proxy can connect to {es} through the firewall. | ||
|
||
Support is available in {agent} and {fleet} for connections through HTTP Connect | ||
(HTTP 1 only) and Socks5 proxy servers. | ||
|
||
This guide describes where a proxy server is allowed in your deployment and how | ||
to configure proxy settings for {agent} and {fleet}. The steps for deploying the | ||
proxy server itself are beyond the scope of this article. | ||
|
||
NOTE: Some environments require users to authenticate with the proxy. There are | ||
no explicit settings for proxy authentication in {agent} or {fleet}, except the | ||
ability to pass credentials in the URL or as keys/tokens in headers, as | ||
described later. | ||
|
||
[discrete] | ||
[[elastic-agent-proxy-config]] | ||
== When to configure proxy settings | ||
|
||
Configure proxy settings for {agent} when it must connect through a proxy server | ||
to: | ||
|
||
* Download artifacts from `artifacts.elastic.co` for subprocesses or binary | ||
upgrades | ||
* Send data to {es} | ||
* Retrieve agent policies from {fleet-server} | ||
* Retrieve agent policies from {es} (only needed for agents running {fleet-server}) | ||
|
||
image::images/agent-proxy-server.png[Image showing connections between {agent}, {fleet-server}, and {es}] | ||
|
||
If {fleet} is unable to access the Elastic Package Registry because {kib} is | ||
behind a proxy server, you may also need to set the registry proxy URL | ||
in the {kib} configuration. | ||
|
||
image::images/fleet-epr-proxy.png[Image showing connections between {fleet} and the Elastic Package Registry] | ||
|
||
[discrete] | ||
[[host-proxy-env-vars]] | ||
== Set default proxy settings in host environment variables | ||
|
||
Set environment variables on the host to configure default proxy settings. | ||
The {agent} uses host environment settings by default if no proxy settings are | ||
specified elsewhere. You can override host proxy settings later when you | ||
configure the {agent} and {fleet} settings. The following environment variables | ||
are available on the host: | ||
|
||
|=== | ||
|Variable |Description | ||
|
||
|`HTTP_PROXY` | ||
|URL of the proxy server for HTTP traffic. | ||
|
||
|`HTTPS_PROXY` | ||
|URL of the proxy server for HTTPS traffic. | ||
|
||
|`NO_PROXY` | ||
|IP addresses or domain names that should not use the proxy. Supports patterns. | ||
|=== | ||
|
||
The proxy URL can be a complete URL or `host[:port]`, in which case the `http` | ||
scheme is assumed. An error is returned if the value is a different form. | ||
|
||
[discrete] | ||
[[where-to-set-proxy-env-vars]] | ||
=== Where to set proxy environment variables | ||
|
||
The location where you set these environment variables is platform-specific and | ||
based on the system manager you're using. Here are some examples to get you | ||
started. For more information about setting environment variables, refer to the | ||
documentation for your operating system. | ||
|
||
* For Windows services, set environment variables for the service in | ||
the Windows registry. | ||
+ | ||
This PowerShell command sets the | ||
`HKLM\SYSTEM\CurrentControlSet\Services\Elastic Agent\Environment` registry | ||
key, then restarts {agent}: | ||
+ | ||
[source,yaml] | ||
---- | ||
$environment = [string[]]@( | ||
"HTTPS_PROXY=https://proxy-hostname:proxy-port", | ||
"HTTP_PROXY=http://proxy-hostname:proxy-port" | ||
) | ||
Set-ItemProperty "HKLM:SYSTEM\CurrentControlSet\Services\Elastic Agent" -Name Environment -Value $environment | ||
Restart-Service "Elastic Agent" | ||
---- | ||
|
||
* For Linux services, the location depends on the distribution you're using. | ||
For example, you can set environment variables in: | ||
|
||
** `/etc/systemd/system/elastic-agent.service` for systems that use `systemd` to | ||
manage the service. To edit the file, run: | ||
+ | ||
[source,shell] | ||
---- | ||
sudo systemctl edit --full elastic-agent.service | ||
---- | ||
+ | ||
Then add the environment variables under `[Service]` | ||
+ | ||
[source,shell] | ||
---- | ||
[Service] | ||
Environment="HTTPS_PROXY=https://my.proxy:8443" | ||
Environment="HTTP_PROXY=http://my.proxy:8080" | ||
---- | ||
|
||
** `/etc/sysconfig/elastic-agent` for Red Hat-like distributions that don't use | ||
`systemd`. | ||
|
||
** `/etc/default/elastic-agent` for Debian and Ubuntu distributions that don't | ||
use `systemd`. | ||
+ | ||
For example: | ||
+ | ||
[source,shell] | ||
---- | ||
HTTPS_PROXY=https://my.proxy:8443 | ||
HTTP_PROXY=http://my.proxy:8080 | ||
---- | ||
|
||
After adding environment variables, restart the service. | ||
|
||
[discrete] | ||
[[proxy-settings-in-agent-policy]] | ||
== Set {agent} proxy settings in the agent policy | ||
|
||
Proxy settings in the {agent} policy override proxy settings specified by | ||
environment variables. This means you can specify proxy settings for {agent} | ||
that are different from host or system-level environment settings. The following | ||
proxy settings are valid in the agent policy: | ||
|
||
|=== | ||
|Setting | Description | ||
|
||
|`proxy_url` | ||
| (string) URL of the proxy server. If set, the configured URL is used as a | ||
proxy for all connection attempts by the component. The value may be either a | ||
complete URL or a `host[:port]`, in which case the `http` scheme is assumed. If | ||
a value is not specified through the configuration, then proxy environment | ||
variables are used. The URL accepts optional `username` and `password` settings | ||
for authenticating with the proxy. For example: | ||
`http://<username>:<password>@<proxy host>/`. | ||
|
||
|`proxy_headers` | ||
| (string) Additional headers to send to the proxy during CONNECT requests. You | ||
can use this setting to pass keys/tokens required for authenticating with the | ||
proxy. | ||
|
||
|`proxy_disable` | ||
| (boolean) If set to `true`, all proxy settings, including the `HTTP_PROXY` and | ||
`HTTPS_PROXY` environment variables, are ignored. | ||
|
||
|=== | ||
|
||
[discrete] | ||
=== Set the proxy for communicating with {es} | ||
|
||
To set the proxy for communicating with {es}, specify proxy settings under | ||
`outputs` in the agent policy. The procedure for doing this depends on | ||
whether you're running {fleet}-managed or standalone agents: | ||
|
||
* For {fleet}-managed agents, specify proxy settings in the {kib} UI: | ||
+ | ||
-- | ||
. Log in to {kib} and go to *Management > {fleet}*. | ||
|
||
. Click *{fleet} settings*. | ||
|
||
. Under *Elasticsearch output configuration (YAML)*, specify proxy settings for | ||
connecting to {es}. The proxy settings you specify here are applied to all | ||
{agent}s enrolled in {fleet}. | ||
+ | ||
[role="screenshot"] | ||
image::images/fleet-proxy-settings-ui.png[Screen capture of Fleet settings UI showing proxy_url setting] | ||
-- | ||
|
||
* For standalone agents, specify proxy settings the `elastic-agent.yml` file. For | ||
example: | ||
+ | ||
[source,yaml] | ||
---- | ||
outputs: | ||
default: | ||
api_key: API-KEY | ||
hosts: | ||
- https://10.0.1.2:9200 | ||
proxy_url: http://10.0.1.7:3128 | ||
type: elasticsearch | ||
---- | ||
|
||
For more information, refer to <<elastic-agent-configuration>>. | ||
|
||
[discrete] | ||
=== Set the proxy for downloading artifacts | ||
|
||
experimental::[] | ||
|
||
To set the proxy for downloading artifacts or binary upgrades from | ||
`artifacts.elastic.co`, add proxy settings under `agent.download` in the | ||
either the `fleet.yml` file for {fleet}-managed agents, or | ||
the `elastic-agent.yml` file for standalone agents. | ||
|
||
For example: | ||
|
||
[source,yaml] | ||
---- | ||
agent: | ||
id: fe277816-8736-4871-9de0-4850d0af21e5 | ||
download.proxy_url: http://10.0.1.3:3128 | ||
logging.level: info | ||
---- | ||
|
||
IMPORTANT: There is currently no way to configure the `agent.download.proxy_url` | ||
setting through a command-line flag or the {fleet} UI. If this capability is | ||
added in the future, the setting you specify here will be overridden. | ||
|
||
[discrete] | ||
[[cli-proxy-settings]] | ||
== Set the proxy for retrieving agent policies from {fleet} | ||
|
||
If there is a proxy between {agent} and {fleet}, specify proxy settings on the | ||
command line when you install {agent} and enroll in {fleet}. The settings you | ||
specify at the command line are added to the `fleet.yml` file installed on the | ||
system where the {agent} is running. | ||
|
||
NOTE: If {kib} is behind a proxy server, you'll still need to | ||
<<epr-proxy-setting,configure {kib} settings>> to access the package registry. | ||
|
||
The `enroll` and `install` commands accept the following flags: | ||
|
||
|=== | ||
| CLI flag | Description | ||
|
||
|`--proxy-url <url>` | ||
|URL of the proxy server. The value may be either a complete URL or a | ||
`host[:port]`, in which case the http scheme is assumed. The URL accepts optional | ||
username and password settings for authenticating with the proxy. For example: | ||
`http://<username>:<password>@<proxy host>/`. | ||
|
||
|`--proxy-disabled` | ||
|If specified, all proxy settings, including the `HTTP_PROXY` and `HTTPS_PROXY` | ||
environment variables, are ignored. | ||
|
||
|`--proxy-header <header name>=<value>` | ||
|Additional header to send to the proxy during CONNECT requests. Use the | ||
`--proxy-header` flag multiple times to add additional headers. You can use | ||
this setting to pass keys/tokens required for authenticating with the proxy. | ||
|
||
|=== | ||
|
||
For example: | ||
|
||
[source,sh] | ||
---- | ||
elastic-agent install -f --url="https://10.0.1.6:8220" --enrollment-token=TOKEN --proxy-url="http://10.0.1.7:3128" --fleet-server-es-ca="/usr/local/share/ca-certificates/es-ca.crt" --certificate-authorities="/usr/local/share/ca-certificates/fleet-ca.crt" | ||
---- | ||
|
||
NOTE: This command requires default policies to be loaded in {fleet}. Default | ||
policies are loaded automatically when you visit {fleet} for the first time. If | ||
you're not sure whether default policies are loaded, log in to {kib} and go to | ||
*Management > {fleet}*. | ||
|
||
The command in the previous example adds the following settings to the | ||
`fleet.yml` policy on the host where {agent} is installed: | ||
|
||
[source,yaml] | ||
---- | ||
fleet: | ||
enabled: true | ||
access_api_key: API-KEY | ||
hosts: | ||
- https://10.0.1.6:8220 | ||
ssl: | ||
verification_mode: "" | ||
certificate_authorities: | ||
- /usr/local/share/ca-certificates/es-ca.crt | ||
renegotiation: never | ||
timeout: 10m0s | ||
proxy_url: http://10.0.1.7:3128 | ||
reporting: | ||
threshold: 10000 | ||
check_frequency_sec: 30 | ||
agent: | ||
id: "" | ||
---- | ||
|
||
[discrete] | ||
[[epr-proxy-setting]] | ||
== Set the proxy URL of the Elastic Package Registry | ||
|
||
{fleet} might be unable to access the Elastic Package Registry because {kib} is | ||
behind a proxy server. | ||
|
||
Also your organization might have network traffic restrictions that prevent {kib} | ||
from reaching the public Elastic Package Registry endpoints, like | ||
https://epr.elastic.co/[epr.elastic.co], to download package metadata and | ||
content. You can route traffic to the public endpoint of EPR through a network | ||
gateway, then configure proxy settings in the | ||
{kibana-ref}/fleet-settings-kb.html[{kib} configuration file], `kibana.yml`. For | ||
example: | ||
|
||
[source,yaml] | ||
---- | ||
xpack.fleet.registryProxyUrl: your-nat-gateway.corp.net | ||
---- | ||
|
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters