As part of my testing process, I deploy a cloud formation stack, run a bunch of test jobs against it, and then delete that stack. In the happy path, everything is fine, but if a test job fails, I can’t retry it because the delete job runs immediately (or, i could configure it to not run on failure, but then I’d end up with a whole bunch of rogue infra).
For a while, I had a job in place which would emit a block step if tests failed so delete wouldn’t run until I unblocked it, but this has the same problem as above: it’s too hard to keep track of which jobs to unblock.
In my ideal scenario, I’d want the delete job to run automatically, say, an hour after a test failed. That would give me time to retry but wouldn’t require me to remember to unblock everything. Also, it would be helpful to be able to mark certain jobs as not do-not-cancel-if-another-build-starts. I always want the delete job to run, even if the build gets cancelled because another commit got pushed.
