This repository has been archived by the owner on Mar 21, 2024. It is now read-only.

Enable the new recovery functionality of PL 1.5 #627

Open

ant0nsc opened this issue Jan 5, 2022 · 1 comment · Fixed by #633

Assignees

Contributor

ant0nsc commented Jan 5, 2022 •

edited by azure-boards bot

Loading

ant0nsc self-assigned this

Contributor Author

ant0nsc commented Jan 10, 2022

Functionality relies on receiving a SIGTERM signal from the environment. AzureML does not send such a signal at present.

Can mimic that by sending by triggering the flag manually in the middle of an epoch, for example.

ant0nsc linked a pull request

that will close this issue

Improve recovery of preempted jobs #633

Merged

6 tasks

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.