Elastic-agent snapshot lookup will use proxy settings

[michel-laterman]

What does this PR do?

Change the HTTP client from the standard library client to one built
from the http settings supplied to the artifact downloader
(agent.download in fleet.yml) so that proxy settings are used for the
initial request to find the artifact location as well as the subsequent
download.

Why is it important?

Allows agent to upgrade to snapshot versions when agents are behind an Internet proxy.

Checklist

• My code follows the style guidelines of this project
• [] I have commented my code, particularly in hard-to-understand areas
• [] I have made corresponding changes to the documentation
• [] I have made corresponding change to the default configuration files
• [] I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

1. Deploy agent behind an Internet proxy.
2. Add a agent.download.proxy_url setting to fleet.yml.
3. Trigger an upgrade to a newer snapshot version through the Fleet UI

Related issues

• Closes Elastic-Agent snapshot downloader ignores proxy settings #27903

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2021-09-13T22:24:23.086+0000

• Duration: 79 min 53 sec

• Commit: f8410a7

Test stats 🧪

Test	Results
Failed	0
Passed	7044
Skipped	16
Total	7060

Trends 🧪

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test	Results
Failed	0
Passed	7044
Skipped	16
Total	7060

[EricDavisX]

PR looks fairly simple and contained - this is great. Still, is there any benefit or simple place to add in a test for the new config? I guess that would entail possibly adding in more error handling to the code base too. Might be particularly needed if the values in the config are not formatted correctly or if the url exists but the artifact isn't found?

[michel-laterman]

i've tested that the elastic-agent can now (correctly) download snapshot artifacts when behind a proxy.

[elasticmachine]

Pinging @elastic/agent (Team:Agent)

[blakerouse]

Looks good.

[michel-laterman]

@EricDavisX, my fix here is to apply an existing config option to another request when retrieving the snapshot artifacts. Config validation already has an issue made for tracking it (#26685), we can also ensure that url values are correct as part of that issue.
I think if the artifact is not found (either by the search or subsequent request), the error is passed up and logged (and i believe that the agent will appear as unhealthy or reconfiguring for a while before aborting the update).