I have a use case for targeting specific pipelines using specific pull request tags. For example, I want all pipelines to default to using our production CI agents, and allow the user to specify a ci:environment:<environment> PR tag to target our <environment> (e.g. staging) CI agents (hosted in an isolated AWS account).
I want to combine the RegExp operator with the array includes operator, and wondering if this is possible? For example, something like the following:
!(build.pull_request_labels includes /^ci:environment:\w+$/)
This would mean that PRs without a specific ci:environment:<value> label always ran in the production cluster, and any that target a specific environment run in their respective cluster(s).
Apologies if this is already possible; I couldn’t see reference to combining RegExp operators and arrays in this way in the docs.
Thanks for the question - welcome to the Buildkite community. Hope you’ve had a great Easter holiday season and hope you’ve been well!
Buildkite supports conditionals in pipelines, - which from what you’ve described can be leveraged to achieve the behaviour you’re looking for. Command steps can be specified with an if key that determines whether the step will be omitted if it’s evaluated as false. In your case with pull request labels: something like the below will omit the step if the build doesn’t include a relevant ^ci:environment:\w+$ label, and from which is sent to Buildkite by the relevant webhook from VCS:
if: build.pull_request.labels includes /^ci:environment:\w+$/
On your second portion - queues and pipelines are isolated within a clustered setup. If you have seperate queues inside the same cluster that are targeting production/dev or lower environment agents, these steps could thus target these with the agentskey at the step level that would additionally use this conditional to run on the relevant production agents. On the other hand (assuming this is the case) - if the production cluster is isolated in entirety from others - you’d most likely need to approach this with a dual pipeline setup whereby both are configured with the same repository, but differ in the cluster they belong to - and are triggered by certain events only (for example only on PR events with Branch Limiting for the production pipeline within a production cluster, and on commitsPRs on the development pipeline within the development cluster etc).
Thanks @james.s, great to hear you can combine the includes operator with a RegExp. Makes the filter quite powerful!
For those who see this, my final conditional is below for our production CI cluster’s pipeline(s):
(!(build.pull_request.labels includes /^ci:environment:\w+$/i) && build.message !~ /^\[ci:environment:\w+\]$/) || (build.pull_request.labels includes /^ci:environment:production$/ || build.message =~ /\[ci:environment:production\]/i)
This picks up branches that either have no ci:environment<name>tag (or [ci:environment:<name>] text in the commit), as well as PRs with a specific ci:environment:production PR tag (or related commit message text).
This allows the developers to target our other, isolated staging pipelines using the ci:environment:staging label / commit text, and it will only be built on those pipelines.
If they require it to be built on both the production and staging pipelines, they can add both the ci:environment:production and ci:environment:staging PR tags / commit text and will be be built on both.
Very useful for us, as we do tinker quite a bit with our CI clusters to get the most performance out of them, improve security, and update CI pckages that could impact the builds.
