Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document best practice configuration for memory limiter with ballast #4201

Closed
pavolloffay opened this issue Oct 15, 2021 · 3 comments
Closed

Comments

@pavolloffay
Copy link
Member

Is your feature request related to a problem? Please describe.

The memory limiter readme https://github.com/open-telemetry/opentelemetry-collector/blob/main/processor/memorylimiterprocessor/README.md mentions that the ballast should be around 1/3 to 1/2 of the memory allocated to the collector. However it does not say if there is any relation between ballast and memory limiter

The helm chart sets limit_percentage to 80% of the total memory, spike_limit_percentage to 25% and ballast to 40%.
This means that the GC runs roughly at the same time when the memory limiter kicks in. https://github.com/open-telemetry/opentelemetry-helm-charts cc) @dmitryax

Describe the solution you'd like

Documentation that clarifies the relation between ballast and memory limiter setting.

Describe alternatives you've considered

Use what the https://github.com/open-telemetry/opentelemetry-helm-charts does.

Additional context

@sanjaygopinath89
Copy link

Looking for a similar doc, which give more details on memory limit and also the relation between ballast and memory limiter.

Have few question related to this .

  1. The limit_percentage value is calculated based on the resource limit given in K8s ..? Similar to how its done in ballast , using value from memory.limit_in_bytes as totalMemory and by using "size_in_percentage * totalMemory / 100"
  2. memory limiter in a pipeline will check the memory usage of entire otel-collector rt ..? or is it specific to pipeline memory usage ? .
  3. If we use a memory limit configuration like given in sample , If otel-collector reach 20% of the configured limit of the container it will start dropping the data rt ..? for example, if otel collector container has 1GB memory limit and with below configuration we will see data drop if the memory utilization is above 200MB rt ..?
    processors: memory_limiter: check_interval: 1s limit_percentage: 50 spike_limit_percentage: 30

@atoulme
Copy link
Contributor

atoulme commented Dec 18, 2023

See #8343 for next steps on this issue. We will likely move to remove the memory_ballast extension.

@mx-psi
Copy link
Member

mx-psi commented Sep 4, 2024

The ballast extension was removed, so I am closing this as wontfix

@mx-psi mx-psi closed this as not planned Won't fix, can't repro, duplicate, stale Sep 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants