-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bulk_indexer: track all response status codes #177
Conversation
Updates the bulk_indexer to track all response status codes and report them appropriately as metrics. Also refactors tests to make them more succinct. Signed-off-by: Marc Lopez Rubio <marc5.12@outlook.com>
type errorFlushFailed struct { | ||
resp string | ||
statusCode int | ||
tooMany bool | ||
clientError bool | ||
serverError bool |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nice 👍🏼
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
one other option would be to have a typed error for each case, and then wrapping them into an error returned on the specific branch, but we can always change that later
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks 👍🏼
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit to test number of docs != requests to ensure that the right unit is used.
bulk_indexer.go
Outdated
if res.StatusCode == http.StatusTooManyRequests { | ||
return resp, errorTooManyRequests{res: res} | ||
e := errorFlushFailed{resp: res.String(), statusCode: res.StatusCode} | ||
if res.StatusCode >= 400 { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Signed-off-by: Marc Lopez Rubio <marc5.12@outlook.com>
bulk_indexer.go
Outdated
if res.StatusCode == http.StatusTooManyRequests { | ||
return resp, errorTooManyRequests{res: res} | ||
e := errorFlushFailed{resp: res.String(), statusCode: res.StatusCode} | ||
if res.StatusCode >= 400 { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: can we have the switch case here as well? I find it more readable than nested if. Here we'll get the added benefit of fewer return statements.
Signed-off-by: Marc Lopez Rubio <marc5.12@outlook.com>
[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [github.com/elastic/go-docappender/v2](https://github.com/elastic/go-docappender) | `v2.1.4` -> `v2.2.0` | [![age](https://developer.mend.io/api/mc/badges/age/go/github.com%2felastic%2fgo-docappender%2fv2/v2.2.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/go/github.com%2felastic%2fgo-docappender%2fv2/v2.2.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/go/github.com%2felastic%2fgo-docappender%2fv2/v2.1.4/v2.2.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/go/github.com%2felastic%2fgo-docappender%2fv2/v2.1.4/v2.2.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Release Notes <details> <summary>elastic/go-docappender (github.com/elastic/go-docappender/v2)</summary> ### [`v2.2.0`](https://github.com/elastic/go-docappender/releases/tag/v2.2.0) [Compare Source](https://github.com/elastic/go-docappender/compare/v2.1.4...v2.2.0) ##### What's Changed - fix: update CODEOWNERS by [@​kruskall](https://github.com/kruskall) in [elastic/go-docappender#181 - bulk_indexer: track all response status codes by [@​marclop](https://github.com/marclop) in [elastic/go-docappender#177 - build(deps): bump github.com/klauspost/compress from 1.17.8 to 1.17.9 by [@​dependabot](https://github.com/dependabot) in [elastic/go-docappender#183 - add otel tracing support by [@​endorama](https://github.com/endorama) in [elastic/go-docappender#182 ##### New Contributors - [@​endorama](https://github.com/endorama) made their first contribution in [elastic/go-docappender#182 **Full Changelog**: elastic/go-docappender@v2.1.4...v2.2.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - "on tuesday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/open-telemetry/opentelemetry-collector-contrib). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40MjUuMSIsInVwZGF0ZWRJblZlciI6IjM3LjQyNS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJkZXBlbmRlbmNpZXMiLCJyZW5vdmF0ZWJvdCJdfQ==--> --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: opentelemetrybot <107717825+opentelemetrybot@users.noreply.github.com> Co-authored-by: Yang Song <songy23@users.noreply.github.com>
Testing notes✔️ test-plan-ok No change in monitoring metrics (as in Setup: Tested with local apm-server with instrumentation 8.15.0 vs upcoming 8.15.1 (8.15 branch) with mocked ES, sending to ESS apm-server mock ES code```go ``` Steps:
8.15.0queryPOST /metrics-*/_search
{
"track_total_hits": false,
"fields": [
{
"field": "@timestamp",
"format": "strict_date_optional_time"
},
{
"field": "labels.status"
},
{
"field": "elasticsearch.events.processed"
},
{
"field": "service.environment"
}
],
"size": 10,
"version": true,
"_source": false,
"query": {
"bool": {
"must": [],
"filter": [
{
"exists": {
"field": "labels.status"
}
},
{
"term": {
"service.environment": "8.15.0"
}
}
],
"should": [],
"must_not": []
}
}
} result{
"took": 9,
"timed_out": false,
"_shards": {
"total": 23,
"successful": 23,
"skipped": 0,
"failed": 0
},
"hits": {
"max_score": 0,
"hits": [
{
"_index": ".ds-metrics-apm.app.apm_server-default-2024.09.02-000001",
"_id": "BdZ0s5EBNgfusvAWuylZ",
"_version": 1,
"_score": 0,
"fields": {
"service.environment": [
"8.15.0"
],
"@timestamp": [
"2024-09-02T15:57:26.531Z"
],
"elasticsearch.events.processed": [
4
],
"labels.status": [
"Success"
]
}
},
{
"_index": ".ds-metrics-apm.app.apm_server-default-2024.09.02-000001",
"_id": "wNZzs5EBNgfusvAW0Cj2",
"_version": 1,
"_score": 0,
"fields": {
"service.environment": [
"8.15.0"
],
"@timestamp": [
"2024-09-02T15:56:26.526Z"
],
"elasticsearch.events.processed": [
4
],
"labels.status": [
"Success"
]
}
},
{
"_index": ".ds-metrics-apm.app.apm_server-default-2024.09.02-000001",
"_id": "e9Zys5EBNgfusvAW5iiZ",
"_version": 1,
"_score": 0,
"fields": {
"service.environment": [
"8.15.0"
],
"@timestamp": [
"2024-09-02T15:55:26.520Z"
],
"elasticsearch.events.processed": [
4
],
"labels.status": [
"Success"
]
}
},
{
"_index": ".ds-metrics-apm.app.apm_server-default-2024.09.02-000001",
"_id": "N9Zxs5EBNgfusvAW1ShN",
"_version": 1,
"_score": 0,
"fields": {
"service.environment": [
"8.15.0"
],
"@timestamp": [
"2024-09-02T15:54:26.515Z"
],
"elasticsearch.events.processed": [
4
],
"labels.status": [
"Success"
]
}
},
{
"_index": ".ds-metrics-apm.app.apm_server-default-2024.09.02-000001",
"_id": "8tZws5EBNgfusvAW6ifm",
"_version": 1,
"_score": 0,
"fields": {
"service.environment": [
"8.15.0"
],
"@timestamp": [
"2024-09-02T15:53:26.509Z"
],
"elasticsearch.events.processed": [
4
],
"labels.status": [
"Success"
]
}
},
{
"_index": ".ds-metrics-apm.app.apm_server-default-2024.09.02-000001",
"_id": "rdZws5EBNgfusvAWACeC",
"_version": 1,
"_score": 0,
"fields": {
"service.environment": [
"8.15.0"
],
"@timestamp": [
"2024-09-02T15:52:26.504Z"
],
"elasticsearch.events.processed": [
4
],
"labels.status": [
"Success"
]
}
},
{
"_index": ".ds-metrics-apm.app.apm_server-default-2024.09.02-000001",
"_id": "aNZvs5EBNgfusvAWFicc",
"_version": 1,
"_score": 0,
"fields": {
"service.environment": [
"8.15.0"
],
"@timestamp": [
"2024-09-02T15:51:26.498Z"
],
"elasticsearch.events.processed": [
4
],
"labels.status": [
"Success"
]
}
},
{
"_index": ".ds-metrics-apm.app.apm_server-default-2024.09.02-000001",
"_id": "I9Zus5EBNgfusvAWUie6",
"_version": 1,
"_score": 0,
"fields": {
"service.environment": [
"8.15.0"
],
"@timestamp": [
"2024-09-02T15:50:26.492Z"
],
"elasticsearch.events.processed": [
14
],
"labels.status": [
"Success"
]
}
},
{
"_index": ".ds-metrics-apm.app.apm_server-default-2024.09.02-000001",
"_id": "yNZts5EBNgfusvAWaCZX",
"_version": 1,
"_score": 0,
"fields": {
"service.environment": [
"8.15.0"
],
"@timestamp": [
"2024-09-02T15:49:26.486Z"
],
"elasticsearch.events.processed": [
4
],
"labels.status": [
"Success"
]
}
},
{
"_index": ".ds-metrics-apm.app.apm_server-default-2024.09.02-000001",
"_id": "f9Zss5EBNgfusvAWfSb3",
"_version": 1,
"_score": 0,
"fields": {
"service.environment": [
"8.15.0"
],
"@timestamp": [
"2024-09-02T15:48:26.479Z"
],
"elasticsearch.events.processed": [
8
],
"labels.status": [
"Success"
]
}
}
]
}
} 8.15.1queryPOST /metrics-*/_search
{
"track_total_hits": false,
"fields": [
{
"field": "@timestamp",
"format": "strict_date_optional_time"
},
{
"field": "labels.status"
},
{
"field": "elasticsearch.events.processed"
},
{
"field": "service.environment"
}
],
"size": 10,
"version": true,
"_source": false,
"query": {
"bool": {
"must": [],
"filter": [
{
"exists": {
"field": "labels.status"
}
},
{
"term": {
"service.environment": "8.15.1"
}
}
],
"should": [],
"must_not": []
}
}
} result{
"took": 4,
"timed_out": false,
"_shards": {
"total": 23,
"successful": 23,
"skipped": 0,
"failed": 0
},
"hits": {
"max_score": 0,
"hits": [
{
"_index": ".ds-metrics-apm.app.apm_server-default-2024.09.02-000001",
"_id": "69Z6s5EBNgfusvAWuCqd",
"_version": 1,
"_score": 0,
"fields": {
"service.environment": [
"8.15.1"
],
"@timestamp": [
"2024-09-02T16:04:08.997Z"
],
"elasticsearch.events.processed": [
5
],
"labels.status": [
"FailedClient"
]
}
},
{
"_index": ".ds-metrics-apm.app.apm_server-default-2024.09.02-000001",
"_id": "7NZ6s5EBNgfusvAWuCqd",
"_version": 1,
"_score": 0,
"fields": {
"service.environment": [
"8.15.1"
],
"@timestamp": [
"2024-09-02T16:04:08.997Z"
],
"elasticsearch.events.processed": [
4
],
"labels.status": [
"FailedServer"
]
}
},
{
"_index": ".ds-metrics-apm.app.apm_server-default-2024.09.02-000001",
"_id": "7dZ6s5EBNgfusvAWuCqd",
"_version": 1,
"_score": 0,
"fields": {
"service.environment": [
"8.15.1"
],
"@timestamp": [
"2024-09-02T16:04:08.997Z"
],
"elasticsearch.events.processed": [
10
],
"labels.status": [
"Success"
]
}
}
]
}
} |
Updates the bulk_indexer to track all response status codes and report them appropriately as metrics. Also refactors tests to make them more succinct.
Closes #172