Ignoring a failing step?

Features Requests
1

Is there a way to define a step that is allowed to fail? It would come in handy to run certain sanity checks, like linting or static code analysis.
It’s important to create visibility on these steps, but they are not necessarily blocking.

I know there is the continue_on_failure on wait steps, but it will not work for this use case for example:

- command: "lint.sh"
- wait: ~
  continue_on_failure: true
- command: "echo Building code"
- wait
- command: "echo Deploying code"

If the linting failed the deployment will never happen, even if the failure wasn’t exactly blocking.

2

Hey there @JoniV — thanks very much for posting and welcome to the forum :slight_smile:

Good news on that front, we totally agree and I’ve actually nearly finished adding support for this. There will be a changelog up about it pretty soon, but essentially you will be able to allow steps to “soft-fail”. Soft failed steps are still failures, they just don’t cause the whole pipeline to fail too, and are opted into via listing allowed exit statuses.

Cheers,
Justin

1 Like
3

Fantastic. I’ll keep an eye on that.

4

I know this isn’t exactly what you want, but another way to get around this for now would be to add || true

- command: "lint.sh || true" # ignore failures
- wait
- command: "echo Building code"
- wait
- command: "echo Deploying code"

Of course that will then make it appear as if the step isn’t failing, but maybe it would be acceptable. If you wanted to surface lint errors more, you could also use buildkite-agent annotate to make note of the hidden failure.

1 Like
5

Thank you, that’s another work-around.

I had the script return exit 0, because I still wanted to make sure some of the steps returned failures but it’s getting a bit noisy.

6

Just a quick update here so you don’t think I’ve forgotten :upside_down_face: — this is still coming, just got delayed a little last week. If the stars align I’ll hopefully get this out the door this week.

3 Likes
7

Fantastic to hear you’re working on a soft fail like feature. We’re very keen for this functionality - the workarounds listed aren’t ideal. Our scenario is something like this:

- command: "bash frontend-ui-tests.sh" # will sometimes fail
- wait
- block: "deploy to staging" # allow us to unblock manually even if last failed step
1 Like
8

Hey everyone :wave:t3: apologies for the radio silence on this one, it unfortunately slipped through somehow and I forgot to get back here to let everyone know that we actually shipped this feature a little while ago! :tada::balloon:

1060×482

1 Like
9

Hello! I am revisiting this topic because my team needs the ability to ignore a failing step in their build. The soft_fail feature works in buildkite exactly how I would like it to; however, the reporting of the results does not translate to the GitHub checks UI to report the failure as we would like. The check says “soft fail” but shows as passing the check.

I have posted about this previously here Soft fail status in github checks but would appreciate another perspective on the desired behavior for the step-level checks status shown in GitHub for soft failures. My dev team would like to see the failure in GitHub because otherwise they do not go out to the Buildkite UI. Specifically, they want the step-level check in GitHub to fail.

10

Hi!

By design, sof_fail is translated as passed in Github commit status.

The purpose of “soft fail” is that “we don’t really care about the outcome, and if it fails, just let me know, but don’t let it get in the way” and the way that translates in GitHub land is with “success”. So we decided to translate it as passed but adding a note that it was soft_fail.

11

Thanks for the quick reply Paula! I see you’re saying that this is the intended behavior. In my case, the github check succeeding with the “soft fail” commit status is not enough to let the developers know that something has failed. So the end result of the current behavior is that they don’t know about the failure.

I see this as not meeting the goals of the soft fail behavior as you stated:
“we don’t really care about the outcome, and if it fails, just let me know, but don’t let it get in the way”

The failed check in github would not get in their way because they can configure which checks are necessary in order to merge.

12

Thanks for the extra detail! Unfortunately, this is a complex case where it’s difficult to translate a “Buildkite concept” to a “GitHub commit status concept”, and our built-in behavior is our gold standard (i.e. most widely used) and we leaned in that direction.