Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix TaskTimeoutException regression #251

Merged
merged 9 commits into from
Mar 26, 2024
Merged

Conversation

nazywam
Copy link
Member

@nazywam nazywam commented Mar 25, 2024

#241 changed the base type of TaskTimeoutError from Exception to BaseException. This caused the exception handlers on https://github.com/CERT-Polska/karton/blob/master/karton/core/karton.py#L182 and https://github.com/CERT-Polska/karton/blob/master/karton/core/karton.py#L189 to not catch the timeout and crash the karton service alltogether.

Minimal test case:

from karton.core import Consumer, Task
from time import sleep

class TimeoutTest(Consumer):
    identity = "karton.timeout-test"
    filters = [{}]

    def __init__(self, *args, **kwargs) -> None:
        super().__init__(*args, **kwargs)
        self.task_timeout = 5

    def process(self, task: Task) -> None:
        print(task.headers)
        sleep(6)


if __name__ == "__main__":
    TimeoutTest.main()

pre-patch:

[2024-03-25 13:44:02,642][INFO] Received new task - 066f5c91-5e51-4998-8043-8e62656f329f
{'kind': 'raw', 'origin': 'karton.mwdb', 'quality': 'high', 'receiver': 'karton.timeout-test', 'share_3rd_party': True, 'type': 'sample'}
Traceback (most recent call last):
  File "/home/michal/work/karton-playground/timeout_test.py", line 18, in <module>
    TimeoutTest.main()
  File "/home/michal/work/karton-playground/venv/lib/python3.10/site-packages/karton/core/utils.py", line 133, in newfunc
    return self.func(owner, *args, **kwargs)
  File "/home/michal/work/karton-playground/venv/lib/python3.10/site-packages/karton/core/base.py", line 257, in main
    service.loop()
  File "/home/michal/work/karton-playground/venv/lib/python3.10/site-packages/karton/core/karton.py", line 339, in loop
    self.internal_process(task)
  File "/home/michal/work/karton-playground/venv/lib/python3.10/site-packages/karton/core/karton.py", line 179, in internal_process
    self.process(self.current_task)
  File "/home/michal/work/karton-playground/timeout_test.py", line 14, in process
    sleep(6)
  File "/home/michal/work/karton-playground/venv/lib/python3.10/site-packages/karton/core/utils.py", line 78, in throw_timeout
    raise TaskTimeoutError
karton.core.exceptions.TaskTimeoutError
<system exit>

post-patch:

[2024-03-25 13:43:37,603][INFO] Received new task - 98a5bdcd-2d25-4a20-9946-8f00eff96943
{'kind': 'raw', 'origin': 'karton.mwdb', 'quality': 'high', 'receiver': 'karton.timeout-test', 'share_3rd_party': True, 'type': 'sample'}
[2024-03-25 13:43:42,606][ERROR] Failed to process task - 98a5bdcd-2d25-4a20-9946-8f00eff96943
Traceback (most recent call last):
  File "/home/michal/work/karton-playground/venv/lib/python3.10/site-packages/karton/core/karton.py", line 180, in internal_process
    self.process(self.current_task)
  File "/home/michal/work/karton-playground/timeout_test.py", line 14, in process
    sleep(6)
  File "/home/michal/work/karton-playground/venv/lib/python3.10/site-packages/karton/core/utils.py", line 78, in throw_timeout
    raise TaskTimeoutError
karton.core.exceptions.TaskTimeoutError
[2024-03-25 13:43:42,609][INFO] Received new task - 18d39b77-391c-4c37-b39d-068a46322a8f

@nazywam nazywam requested a review from a team March 25, 2024 12:47
karton/core/karton.py Outdated Show resolved Hide resolved
@psrok1 psrok1 merged commit 1c2bc13 into master Mar 26, 2024
6 checks passed
@psrok1 psrok1 mentioned this pull request Mar 26, 2024
@nazywam nazywam deleted the bugfix/handle-timeoutexception branch March 26, 2024 18:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants