Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pulumi cancel on workflow cancelation #928

Open
blampe opened this issue May 2, 2023 · 1 comment
Open

pulumi cancel on workflow cancelation #928

blampe opened this issue May 2, 2023 · 1 comment
Labels
kind/enhancement Improvements or new features

Comments

@blampe
Copy link
Contributor

blampe commented May 2, 2023

Hello!

  • Vote on this issue by adding a 👍 reaction
  • If you want to implement this feature, comment to let us know (we'll work with you on design, scheduling, etc.)

Issue details

Sometimes a user might cancel workflow steps because they no longer need to run them, or because they know they will fail for some reason. If one of these steps happens to be in the middle of a Pulumi update, it will leave the stack in a locked state until someone manually restores it by running pulumi cancel.

This is inconvenient, and we should be able to do this automatically because the workflow cancelation sends us a SIGINT. We have 10 seconds to catch that and call pulumi cancel. From the docs:

For steps that need to be canceled, the runner machine sends SIGINT/Ctrl-C to the step's entry process (node for javascript action, docker for container action, and bash/cmd/pwd when using run in a step). If the process doesn't exit within 7500 ms, the runner will send SIGTERM/Ctrl-Break to the process, then wait for 2500 ms for the process to exit. If the process is still running, the runner kills the process tree.

Affected area/feature

@blampe blampe added needs-triage Needs attention from the triage team kind/enhancement Improvements or new features labels May 2, 2023
@justinvp justinvp removed the needs-triage Needs attention from the triage team label May 3, 2023
@mikocot
Copy link
Contributor

mikocot commented Mar 5, 2024

Even bigger problem is that in many cases cancellation ends up with pulumi dropping resource ccreation in the middle. Resource is then created by the provider, but not added to state. In most cases that effects with zombie resources that can't be removed, yet blocking parent resources from being cleaned up, and blocking any upsert restart attempt. Refresh might help, but only if the name is fixed, not randomized. If we could improve that, it would be a massive upgrade. I understand that pulumi process must exit, but pulumi should be registering such tries internally and confirming if resource was in the end created or not (e.g. during refresh)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/enhancement Improvements or new features
Projects
None yet
Development

No branches or pull requests

3 participants