I’d love to be able to set a concurrency limit on a Pipeline, so that I can have X number of builds running at the same time at most, rather than just on a step by step basis.

Yes yes yes! We are going to have to make some hacky workaround where we wrap another build in a job or make our own locking mechanism. It seems like a critical feature to have to only allow one build per pipeline.

Hi folks, Apologies for the late response here, but I believe I have good news! An option which I believe will suit your needs, but I confess we’ve not been very good at explaining is what we call concurrency gating . In short, the jobs which are part of a concurrency group within a given build wh…

Hi jess, Thanks for the quick reply. Sorry it’s taken me a couple of days to respond. I thought I had tested this but it turns out the little experiment I had done with concurrency gating hadn’t quite been showing my I thought it was. I have re-done the experiment and can confirm that it works as y…

I understand that this workaround solves most of my problems but it can not be called a nice solution IMHO. Any hope of a pipeline “native” solution? Only one invocation of the pipeline should be running at any time… not just parts of it.

Hello, @Asmund ! Welcome to the Buildkite Community! Thank you for your feedback! At the moment, the recommended way to ensure that only one build will be running on a given pipeline is to use the concurrency gates . However, as an alternative, you could set up a cluster with a default queue contai…

@karen.sawrey thank you for responding! Or problem is that we end up having multiple release pipelines instances running in parallel. Leading to random concurrency problems. Running the release in sequence would take > 4h. Parallel it is ~20min. Please update this topic if Buildkite ever decide t…

You’re welcome, @Asmund ! From what you’re describing, I’d really suggest you give concurrency gate a go. If you have any difficulties implementing that, you can reach out to us via support@buildkite.com for us to help troubleshoot your specific use case. If down the line we do add some functionali…

Hi @karen.sawrey I did try the above suggestion and it did not work for me. What am I doing wrong? Could it be that the “gate” do not work across a ‘wait’. If so could you provide an example with a ‘wait’ between? --- agents: queue: ci-blue env: BUILDKITE_TOKEN_PATH: /run/secrets/buildkite-ap…

Hello again, @Asmund ! A small tweak should fix things. What’s missing here is the dependencies after opening the concurrency gate. So you can either add a wait step after opening it: - label: ":buildkite:" key: ":init:" concurrency: 1 concurrency_group: "Block-concurrent-releases"…

Pipeline Concurrency Limit

Features Requests

robyoung November 12, 2020, 1:37pm 3

Is there a workaround for this limitation?

Controlling concurrency of multi-step deployment job

Topic		Replies	Views
Question about "Controlling Concurrency" Pipelines	4	594	July 17, 2023
Concurrency Limit Behavior and Concurrency Gates General	3	1080	February 23, 2022
Using concurrency_group for multiple steps General	5	2100	January 19, 2022
Concurrency Gate with dynamically injected block step Pipelines	6	91	January 14, 2025
Group Step Concurrency Group Features Requests	4	274	July 19, 2023

Pipeline Concurrency Limit

Related topics