Hi Buildkite crew,
Background: We are using bors-ng to merge our Github pull requests (PRs). Bors merges one or more PRs at a time in batches and it will not continue with a new batch until a specific Github commit status is set to green or red, or a timeout occurs (generally after 30 min or so).
Currently we have a pipeline consisting of multiple step running in parallel (ie. there is no wait step); a few quick code linters (take 30 seconds) and a very slow unit test step (takes ~10 minutes).
Here’s our issue: If a single developer is merging a pull request which breaks some linting rule, Buildkite will not report back that that the commit is Broken until all parallel steps have finished. This means that Buildkite reports back to Github that the commit is RED after 10 minutes (the time it takes for our tests to finish) even though it knew the build failed within 30 seconds . This is waste to us as it blocks another developers pull request to get merged.
Do you have any plans/options/possibilities to make Buildkite submit that the pipeline has failed even though not all steps have finished? It would make wonders for us. If not, see this as feedback and a feature request.
Our current workaround is that we’ve added a “wait” step between the linting and the unit tests, however, this has the cost of adding yet another 30 seconds to every succesful merge which really also is waste.