Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Research spill prioritization based on thread priority (retry framework) #7710

Open
Tracked by #8026
abellina opened this issue Feb 8, 2023 · 0 comments
Open
Tracked by #8026
Labels
reliability Features to improve reliability or bugs that severly impact the reliability of the plugin

Comments

@abellina
Copy link
Collaborator

abellina commented Feb 8, 2023

This very much feels tied to the OOM framework that we are working on #7253.

The thought here is to make sure that the thread with highest priority has less of a chance to have its buffers spilled. We have some exec nodes that will make spillable and then do some compute, in a chain. If the chain is broken because we run out of memory, the buffers added by this high priority thread should probably be the last ones to spill.

This issue is to visit this as we work on the retry framework.

@mattahrens mattahrens added the reliability Features to improve reliability or bugs that severly impact the reliability of the plugin label Oct 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
reliability Features to improve reliability or bugs that severly impact the reliability of the plugin
Projects
None yet
Development

No branches or pull requests

2 participants