Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[exporter/elasticsearch] Use confighttp.ClientConfig #33367

Merged

Conversation

axw
Copy link
Contributor

@axw axw commented Jun 4, 2024

Description:

Move Authentication out of ClientConfig, and replace ClientConfig with confighttp.ClientConfig. This enables all common confighttp functionality, such as auth extensions, with the exception of "compression".

For now compression remains unconfigurable, and is always enabled and uses gzip (this is defined within the go-docappender dependency). We should add some benchmarks (#32504) before switching to the confighttp-based compressors.

The exporter now implements a component.StartFunc, and we create the bulk indexer there so that we have access to component.Host for calling confighttp.ClientConfig.ToClient.

Link to tracking Issue:

N/A

Testing:

  • Added a test verifying that confighttp's auth is supported, with a mock client auth
  • Performed some manual testing with the basicauth extension, verifying the username/password are passed through to ES as configured

Documentation:

Updated the exporter's README:

  • added mention that confighttp's endpoint setting is now supported too
  • simplified the example configuration
  • removed the confighttp, configtls, and sender_queue settings, and replaced with links to their docs
  • split the configuration settings into categories, with the required ones up front (URL + credentials) and optional ones in multiple sections under "Advanced configuration"

Move Authentication out of ClientConfig, and
replace ClientConfig with confighttp.ClientConfig.
Implement StartFunc, and create bulk indexer there,
so we have access to component.Host for calling
confighttp.ClientConfig.ToClient.

This enables all standard confighttp functionality,
such as auth extensions.
@axw axw force-pushed the elasticsearchexporter-confighttp branch from 511fead to d34e708 Compare June 4, 2024 13:02
@axw
Copy link
Contributor Author

axw commented Jun 4, 2024

#33350 will conflict, so I'll open this up once #33350 is merged.

@axw axw marked this pull request as ready for review June 6, 2024 09:58
@axw axw requested review from a team and fatsheep9146 June 6, 2024 09:58
Copy link
Contributor

@carsonip carsonip left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I scanned through the PR and left a comment about compression. I'll review it again when the PR is ready.

exporter/elasticsearchexporter/README.md Outdated Show resolved Hide resolved
axw and others added 2 commits June 7, 2024 08:40
Co-authored-by: Carson Ip <carsonip@users.noreply.github.com>
@axw axw requested a review from carsonip June 7, 2024 00:57
@axw
Copy link
Contributor Author

axw commented Jun 7, 2024

Thanks! I scanned through the PR and left a comment about compression. I'll review it again when the PR is ready.

@carsonip thanks - the PR is ready for review.

Copy link
Contributor

@codefromthecrypt codefromthecrypt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lookin sharp! added some tidies to consider or dismiss, nothing important.

should we add a test for the round-robin? add a bad host like https://not-a-real-host.com:9200 in front of a working one (setting timeouts to a small value so tests don't take too long)?

.chloggen/elasticsearchexporter-confighttp.yaml Outdated Show resolved Hide resolved
exporter/elasticsearchexporter/README.md Outdated Show resolved Hide resolved
exporter/elasticsearchexporter/README.md Outdated Show resolved Hide resolved
exporter/elasticsearchexporter/README.md Outdated Show resolved Hide resolved
exporter/elasticsearchexporter/README.md Outdated Show resolved Hide resolved
exporter/elasticsearchexporter/README.md Outdated Show resolved Hide resolved
exporter/elasticsearchexporter/README.md Show resolved Hide resolved
exporter/elasticsearchexporter/exporter.go Show resolved Hide resolved
Copy link
Contributor

@carsonip carsonip left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thanks!

axw and others added 2 commits June 12, 2024 08:27
@axw axw requested a review from andrzej-stencel June 12, 2024 03:06
@andrzej-stencel andrzej-stencel merged commit 54d0dcb into open-telemetry:main Jun 12, 2024
158 of 162 checks passed
@github-actions github-actions bot added this to the next release milestone Jun 12, 2024
@axw axw deleted the elasticsearchexporter-confighttp branch June 12, 2024 23:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants