Conditional Delay Step

As part of my testing process, I deploy a cloud formation stack, run a bunch of test jobs against it, and then delete that stack. In the happy path, everything is fine, but if a test job fails, I can’t retry it because the delete job runs immediately (or, i could configure it to not run on failure, but then I’d end up with a whole bunch of rogue infra).

For a while, I had a job in place which would emit a block step if tests failed so delete wouldn’t run until I unblocked it, but this has the same problem as above: it’s too hard to keep track of which jobs to unblock.

In my ideal scenario, I’d want the delete job to run automatically, say, an hour after a test failed. That would give me time to retry but wouldn’t require me to remember to unblock everything. Also, it would be helpful to be able to mark certain jobs as not do-not-cancel-if-another-build-starts. I always want the delete job to run, even if the build gets cancelled because another commit got pushed.

Hi @ianwremmel are you suggesting we have a delayed step type of thing? Something that doesn’t run until after some timeout? I can add a feature suggestion internally for that if you like.

As a workaround for now though, you could configure a scheduled build that can use the API to find builds in the specific state and could unblock them via the API as well - as a way to automate this daily or something similar.

yea, or maybe a property on block that unblocks after a timeout?

If a build fails with not-yet-run blocked steps, will its state be blocked or failed?

No worries. I’ll raise that with the team :+1:

You can configure the blocked_state of block steps: Block Step | Buildkite Documentation