We currently make use of the github_deployment_status:environment metadata property as outlined in the Deployment docs. This is really handy as our environment URL is read dynamically from the terraform state outputs. Neat.
I was wondering if it was possible to also do something similar for the deployment itself (as opposed to the deployment_status), specifically the ability to specify the id of a dynamically generated GitHub deployment resource that we create as part of a command step. For example: buildkite-agent metadata set github_deployment:id <deploymentId>
Example use case:
Currently, our pipeline checks to see if the related PR has a deploy:<environment> tag, and dynamically injects a trigger step to that <environment> for our deploy pipeline. This same pipeline has the “Trigger on GitHub deployments” enabled in the Buildkite UI.
This works relatively well, and it means we can 1) trigger deployments via the UI easily, 2) auto-deploy our build pipelines to specific environments using PR labels, and 3) utilise programmatic calls to GitHub to trigger the deployment pipelines (via client.repos.createDeployment API).
However, one downside of this approach is that it doesn’t automatically notify GitHub of the deployment, as there is no BUILDKITE_GITHUB_DEPLOYMENT_ID on initial pipeline trigger, since the deployment is yet to happen.
It’d be ideal if I could, say, call buildkite-agent metadata set github_deployment:id <deploymentId>, in my initial command step, which would trigger the same behaviour as the initially set BUILDKITE_GITHUB_DEPLOYMENT_ID environment variable.
Alternatively, is there a way to trigger this behaviour in another way that a way that doesn’t require the BUILDKITE_GITHUB_DEPLOYMENT_ID to be set on the pipeline environment?
This is Suma from Buildkite support team. Thank you for reaching out to us with your question.
As you mentioned we do not have the option for github deployment id in metadata so this can be automated. So without that only option is to have in the build steps a script that calls the github create deployment API in order to trigger the deployment pipeline from Buildkite.
I figured that might be the case. Using a command step that triggers a GitHub deployment will works, but sadly it means I lose the connection between the Build => Deployment pipeline, as I’d need to remove the async trigger step. Which also means it’s harder to trace where a deployment originated from directly in the Buildkite UI.
To partially workaround this, I’m calling annotate with the link to the deployment instead, but this only works on the Build pipeline side. Though, I’m curious if I can utilise the follow environment variables that get set by Buildkite on async trigger build to get the UI to display the information again if I propagate it through the deployment?
Sadly, I just tried to set these directly and Buildkite overrides their values, which makes sense. Wouldn’t want people injecting random values here that aren’t vetted as correct.
I think I’ll instead have to do some environment hook logic that extracts this information from the GitHub deployment payload and inject it as environment variables, which I can then use to annotate the Deploy pipeline with the Build pipeline link that triggered it.
Unless you can think of any other ways we can simplify this workflow?
Another option would be triggering pipeline via API, as @suma mentioned, although using environment variables that do not interfere with ones populated by the agent. Trigger Buildkite Pipeline · Actions · GitHub Marketplace · GitHub might be also worth exploring — it allows setting both build_env_vars and build_meta_data which might come handy in this case.
,