Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not keep rerunning all background jobs #278

Open
olevski opened this issue Jun 21, 2024 · 1 comment
Open

Do not keep rerunning all background jobs #278

olevski opened this issue Jun 21, 2024 · 1 comment
Labels
cooldown Issues to be tackled during cooldown

Comments

@olevski
Copy link
Member

olevski commented Jun 21, 2024

The purpose of some background jobs are to simply fix one-time issues that have cropped up in the regular or authorization database. Therefore we do not need to keep rerunning them all the time - as we currently do.

So we need some way to limit how many times a specific job runs.

@leafty suggested that we just keep track in the database on how many times a job has run and then not run it after some number. But is good to run these multiple times to make sure we avoid synchronization problems or race conditions.

@olevski olevski added the cooldown Issues to be tackled during cooldown label Jun 21, 2024
@leafty
Copy link
Member

leafty commented Jun 21, 2024

My suggestion would be that for data migration jobs: at the start of the job collect the set of items to process. If that set is empty N=3 times in a row, you have reached consistency and the job doesn't need to run anymore.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cooldown Issues to be tackled during cooldown
Projects
None yet
Development

No branches or pull requests

2 participants