Notifying on first instance of a pipeline failure?

Pipelines
1

I’m trying to set up a Slack notification so there’s an alert sent on the first instance of a pipeline failing. And then no subsequent alerts sent until the pipeline passes again.

So if there are N failures before the pipeline passes again, I only get two alerts sent: the first time it failed, and then again when it starts to pass.

Under Choose notifications to send... in the Notification Services settings in the the Failed option what I want here? Will that only alert once in this case?

I’m failing to turn up docs that explain the difference between Failed and Failing in this setting screen:

Screenshot 2024-06-25 at 3.59.53 PM
Screenshot 2024-06-25 at 3.59.53 PM1852×1094 156 KB

2

Hey @ian-persona

Here is a link in docs describing the Build States. BuildStates – Enums – GraphQL API | Buildkite Documentation

FAILED : The build failed
FAILING: The build is failing

So you should get notifications when a Build has failed and when there has been fix on the pipeline. Essentially The Failing state is a process that is continuous which means all the job in that particular build have not yet finished.

What i noticed in the notifications is if you select only Fixed and Failed Builds, it will default to only send notifications for failed builds and rebuilds.

A diagram to show Build State transitions in Buildkite

3

@stephanie.atte that’s a helpful state diagram. Thank you!

It would seem that what I’m after isn’t possible then? If I have the following pipeline results:

Passed A - Failed 1 - Failed 2 - Failed 3 - Failed 4 - Passed B

Having only Failed 1 and Passed B send notifications to the Slack channel is what I’m trying to achieve.

This might be a feature request.

4

Hey @ian-persona

Yes exactly we do not have this feature natively. I think you might be able to achieve something custom by leveraging the Builds API to find out the build states in between and then create a notification based on that results.

Another option might be to use trigger step to trigger a pipeline that can send that notification based on the conditions.

5

I looked in to this and I’d need to build an entirely separate service that monitors the pipeline and builds. Not ideal here.

I can’t see how to refer to the state of the build that came before me when writing trigger step logic. Is that possible?

This is all likely a feature request: a way to have the built-in Slack notifications be less noisy.

6

That is correct. For that you will need to have, for example, an artifact that records the previous states. It’s not ideal, and basically, it will require building your own service.

We don’t have another option at the moment, but our Product team will work during next quarter to prioritize and design a better Slack integration, that will hopefully make this type of request easier.