Pre-pipeline / post-pipeline hooks for job lifecycle

We use dynamic pipelines (buildkite-agent pipeline upload) and noticed that each of the steps being run is a separate job, as in every hook defined Buildkite Agent Hooks v3 | Buildkite Documentation executes on every step.

That’s fine… a little interesting to wrap your head around, but that’s okay. But that means I can’t use pre-command and post-command, or even pre-exit because they’ll run 15+ times.

Is there a pre-pipeline hook, and a post-pipeline hook? Would love to run something before/after my builds exactly once.

Hi @chomey that’s right hooks run for each job. To achieve what you wish, could you use a step itself instead of a hook?
The first step in the pipeline (could even define it before the pipeline upload step with a wait step between) could perform your logic. And you could define the last step in the pipeline in a similar way.

Dynamic pipelines all upload their steps after the step that is uploading them, so if you define the final step in the web UI, it won’t have steps added after it by future uploads.

And by allowing steps to continue on failure you can ensure that the last step will always still run, even if previous steps fail.

I suppose I could use a step… but the intent was to have a default set of pre/post pipeline hook functions that ran, regardless of job, so that we didn’t have to define them in every pipeline.

Think notifications to Slack, metrics, or annotating gifs into build outputs =)

Oh yeah that makes sense. Its quite a difficult problem because no code runs on our server so the agent would need to know its running the first/last step. But the last one is difficult because that could change during the execution of that step (if it uploaded more). I’ll raise a feature request for it though to see what the team thinks

I’m trying to be clever using the CLI agent step command. I’m getting this from the pre-exit hook, is this expected?

[2022-05-25T06:43:09Z] {
[2022-05-25T06:43:09Z]   "label": "Publish",
[2022-05-25T06:43:09Z]   "type": "command",
[2022-05-25T06:43:09Z]   "if": null,
[2022-05-25T06:43:09Z]   "state": "running",
[2022-05-25T06:43:09Z]   "key": "end",
[2022-05-25T06:43:09Z]   "outcome": null,
[2022-05-25T06:43:09Z]   "notify": [
[2022-05-25T06:43:09Z]
[2022-05-25T06:43:09Z]   ],
[2022-05-25T06:43:09Z]   "command": "<mycommand redacted>",
[2022-05-25T06:43:09Z]   "parallelism": null,
[2022-05-25T06:43:09Z]   "env": null,
[2022-05-25T06:43:09Z]   "timeout": null,
[2022-05-25T06:43:09Z]   "concurrency_limit": null,
[2022-05-25T06:43:09Z]   "concurrency_key": null,
[2022-05-25T06:43:09Z]   "agents": [
[2022-05-25T06:43:09Z]     "os=linux",
[2022-05-25T06:43:09Z]     "project=testinfrogsproject"
[2022-05-25T06:43:09Z]   ]
[2022-05-25T06:43:09Z] }

I’m surprised “outcome” is still null at this phase?

Hi @chomey

Please can you share more details on what you have setup in pre-exit hook. I assume you are using buildkite-agent step get "outcome" --step "STEPKEY" in your pre-exit hook but you are observing null value for it

Yes, that’s exactly right @suma.

Pipeline:

[2022-05-25T06:42:58Z] steps:
[2022-05-25T06:42:58Z]   - label: "Publish"
[2022-05-25T06:42:58Z]     commands:
[2022-05-25T06:42:58Z]       - mycommand
[2022-05-25T06:42:58Z]     key: "end"

pre-exit hook:

buildkite-agent step get --format "json" --step "end"

Build log:

Running local pre-exit hook
[2022-05-25T06:42:59Z] $ .buildkite/hooks/pre-exit
[2022-05-25T06:43:04Z] {
[2022-05-25T06:43:04Z]   "label": "Publish",
[2022-05-25T06:43:04Z]   "type": "command",
[2022-05-25T06:43:04Z]   "if": null,
[2022-05-25T06:43:04Z]   "state": "running",
[2022-05-25T06:43:04Z]   "key": "end",
[2022-05-25T06:43:04Z]   "outcome": null,
[2022-05-25T06:43:04Z]   "notify": [
[2022-05-25T06:43:04Z]
[2022-05-25T06:43:04Z]   ],
[2022-05-25T06:43:04Z]   "command": "mycommand",
[2022-05-25T06:43:04Z]   "parallelism": null,
[2022-05-25T06:43:04Z]   "env": null,
[2022-05-25T06:43:04Z]   "timeout": null,
[2022-05-25T06:43:04Z]   "concurrency_limit": null,
[2022-05-25T06:43:04Z]   "concurrency_key": null
[2022-05-25T06:43:04Z] }

Hi @chomey

Thank you for confirming. I was able to reproduce the behavior like you said but looking at the documentation and following that where I created pipeline like below with a second step then I could see the pre-exit output of the second step showing proper outcome value

steps:
- label: "Building echos"
  commands:
        - command1
  key: "end"
- label: "outcome"
  commands:
        - command2

Below is the build log where we can see pre-exit hook output showing outcome as “passed”

Running global pre-exit hook
$ /opt/homebrew/etc/buildkite-agent/hooks/pre-exit
{
  "label": "Building echos",
  "type": "command",
  "if": null,
  "state": "finished",
  "key": "end",
  "outcome": "passed",
  "notify": [
 
  ],
  "command": "echo test",
  "parallelism": null,
  "env": null,
  "timeout": null,
  "concurrency_limit": null,
  "concurrency_key": null,
  "agents": null
}

So based on this it looks like outcome value will only be accessible in subsequent step.

Right… which again doesn’t help me when I’m trying to find the last step =)

Yes @chomey. Sorry, we do not have a better answer for your usecase at the moment. We raised feature request with our team on having ability to define a hook which runs as pre-start or pre-exit step for a pipeline instead of running it for every pipeline step.

Hi @chomey

Our product team reviewed the feature request and consider that using plugins for this which you can invoke across pipelines that need it would be better way to handle it. As plugin approach can help with this team considers this should be enough for this usecase.

Here is documentation which explains about how to use plugins in a pipeline

Please let us know if you have any questions. Thank you