Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prep for v0.30.2 #2723

Merged
merged 5 commits into from
Jan 11, 2023
Merged

Conversation

rfratto
Copy link
Member

@rfratto rfratto commented Jan 11, 2023

No description provided.

This commit clones the label set before applying relabels. Not
cloning does two things:

1. It forces the computed ID of the incoming series to change (as its
   labels changed)

2. It can cause obscure bugs with relabel rules being applied, such as
   a `keep` action which doesn't work after modifying the original
   slice.
…grafana#2721)

Fix an issue where being unable to send logs to `loki.write` due to the
client being permanently backlogged would deadlock the Flow controller.

The `loki.write` client may be permanently backlogged when:

* Limits are reached when sending logs to Loki, leading to endless
  request retries.
* Loki has an extended outage.

When an EntryHandler is stopped, it will wait for 5 seconds before
forcibly stopping the goroutine which queues log entries. If this
timeout is reached, any unqueued log entries are permanently lost, as
the positions file will likely be updated past the point where the entry
was read.

While losing logs is not ideal, it's unacceptable for any Flow component
to be able to block the controller. This is a short-term solution to
allow the Flow controller to continue working properly. A long term
solution would be to use a Write-Ahead Log (WAL) for log entries. See
grafana/loki#7993.

Fixes grafana#2716.
Related to grafana/loki#2361.
Copy link
Member

@tpaschalis tpaschalis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@karengermond karengermond left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

CHANGELOG.md Outdated Show resolved Hide resolved
CHANGELOG.md Outdated Show resolved Hide resolved
@rfratto rfratto merged commit 59144c6 into grafana:release-v0.30 Jan 11, 2023
@rfratto rfratto deleted the prep-release-v0.30.2 branch January 11, 2023 17:10
@github-actions github-actions bot added the frozen-due-to-age Locked due to a period of inactivity. Please open new issues or PRs if more discussion is needed. label Mar 13, 2024
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 13, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
frozen-due-to-age Locked due to a period of inactivity. Please open new issues or PRs if more discussion is needed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants