Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: correct end time when a trace is outside the ingestion slack range #3954

Merged
merged 4 commits into from
Aug 12, 2024

Conversation

javiermolinar
Copy link
Contributor

@javiermolinar javiermolinar commented Aug 9, 2024

What this PR does:
This PR addresses a bug where traces older than the ingestion slack are ingested. When that happened, the block start time exceeded the block end time. This led the compactor to remove those blocks, and the inability to search them by id.

Example of the current error
Trace data:

{
          "traceId": "5B8EFFF798038103D269B633813FC703",
          "spanId": "EEE19B7EC3C1B100",
          "name": "I am a span!",
          "startTimeUnixNano": 1689969302000000000,
          "endTimeUnixNano": 1689970000000000000,
          "kind": 2,
          "status": {
            "code": "STATUS_CODE_OK"
          },
          "attributes": [
          {
              "key": "my.span.attr",
              "value": {
                  "stringValue": "some value"
              }
}

The start and date are from 2023

Resulting block:

window: 2023-07-21T20:00:00Z
start: 2024-08-07T17:21:44+02:00
end: 2023-07-21T22:06:40+02:00
duration:  -9187h15m4s
age: 9203h48m32s

The start date is being set as now() whereas the end date is left as it is in the span data.

Example after the PR changes

{"format":"vParquet4","blockID":"be564c9d-f76d-49ee-9668-24b6cc024eed","minID":"W47/95gDgQPSabYzgT/HAw==","maxID":"W47/95gDgQPSabYzgT/HAw==","tenantID":"single-tenant","startTime":"2024-08-09T15:45:16.837762+02:00","endTime":"2024-08-09T15:47:18+02:00","totalObjects":2,"size":0,"compactionLevel":0,"encoding":"none","indexPageSize":0,"totalRecords":0,"dataEncoding":"","bloomShards":0,"footerSize":0}

Now, in the wall block, the start and end times are set to now() and there are no inconsistencies.

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

@javiermolinar javiermolinar changed the title fix: correct end time when a trace is appended to a block if is smaller than start time after the adjustment fix: correct end time when a trace is outside the ingestion slack range Aug 9, 2024
// start/end are unix epoch seconds
func (b *BlockMeta) ObjectAdded(id []byte, start, end uint32) {
// start/end are unix epoch seconds, when 0 the start and the end are not applied.
func (b *BlockMeta) ObjectAdded(start, end uint32) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removing id was missed in the min/max removal? nice cleanup

@joe-elliott joe-elliott merged commit 85a2edb into grafana:main Aug 12, 2024
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants