Several of our pipelines commit back to their respective repo after they have finished running a build and executing tests. The initial commit to the repo from a PR successfully reports a passed status in Github, but the commit back with [skip ci] in the commit message ends up blocking the PR because the status check is required to pass, but Buildkite’s API does not appear to return anything on the webhook call to the API.
In Circle CI, when you use [skip ci], the build is successfully skipped, and Circle returns a ‘neutral’ status for the required status check, which Github then interprets as a pass, allowing the PR to be merged if all merge constraints are met.
This behavior puts us in a situation where we can’t require specific status checks because they block our PRs indefinitely. I sent an email to support@ and messaged in Slack, but my experience with both of those avenues for support has been lacking.
I’ve included a screenshot of the behavior I’m describing.
Hey @krohmag, I saw you messaged us in a few places, will reply here and post a link in the others! Sorry for the slow reply, just ticked over to the weekend for us.
Unfortunately the GitHub Commit Status API doesn’t support a neutral state for commit statuses only error, failure, pending, or success, which limits our options somewhat.
I think what that would mean is that we’d have to send a “success” for that commit, which could potentially be dangerous.
We’ve got plans for more granular control over commit statuses in future beyond just at a build level which might open up some more doors, but I’m afraid I don’t have a good answer for how to send a success commit status for skipped builds presently! One option might be to allow pipelines to opt-out entirely from the [skip ci] behaviour? It feels to me like it doesn’t fit in a world where you have required status checks in a PR.
I’m curious as to how Circle is able to have a ‘neutral’ state. We have a required check in Circle, and if I execute a [skip ci] build, the build is skipped by Circle, and in Github, the build is able to proceed as Github interprets that neutral state as a pass.
In our case, [skip ci] is an ideal solution. When we have a PR, a buildkite build is kicked off, which builds a Docker image, executes tests, and then pushes images upon passing tests. The hash of the resulting Docker image is committed back to the repo by a bot user so deployments use the specific image created by the PR, avoiding any race conditions with other PRs in progress. That commit back does not warrant another build to be kicked off. In fact, another build kicking off would result in an infinite loop of builds, which is certainly less than ideal.
As a means of circumventing this shortcoming/oversight, I’ve modified our build scripts to check if the build creator is the bot user. If it is, a ‘dummy’ build is executed so we can get a passing status check, allowing our PR to be merged. It’s a bit hackey and definitely adds some overhead to our process, but it accomplishes what we need based on our specific use-case.
All that to say, that the current implementation of [skip ci] seems to only pertain to addressing a capacity issue - like one would have in Circle. With Buildkite, we control the number of agents we’re running, so capacity isn’t really as issue, but there are other uses of [skip ci] that do not seem to be addressed in the current Buildkite implementation.
