Hi,
We’re using the soft_fail feature for some steps in our CI pipelines, and it works well on the buildkite side, but the soft failures are reported as successes on GitHub. This old response calls this intended, but we would like an option to report soft failures as failed to GitHub instead, and it would really help us simplify our pipeline.
For context, we have a few different merge target branches, each representing a different release target or release stage in our process. We run a unified pipeline shared between those target branches, with some hacky workarounds to detect which branch’s requirements must be met. Since the requirements for these branches can differ, we’d like instead to have some jobs set to soft_fail, with the goal of requiring them via GitHub’s branch protection rules for the branches that need them. Obviously this can’t work today, since all the step statuses report as success.
Anecdotally too, our developers have missed a few long-term breakages in our soft_fail jobs, so making them visible in GitHub would help us catch those earlier as well. If this feature isn’t too hard or invasive to integrate, we would really appreciate it.
Many thanks,
Matt