Buildkite pipeline priority

Hi Team,

We encountered a situation where we have a limited pool of agents and they were engaged with develop branch builds. So when we ran master branch builds they all went in queue.
So it there a way to the builds based upon the branches ? The prioritization can be using identifies like branch or an env variable.
So idea here is if there are two builds in waiting stage, one from develop branch and other from master branch, the master branch is picked first.

Thanks

Hi Suraj,

Thanks for checking with us here!

If there are multiple builds that are waiting for agents, the jobs that are scheduled first would start to run first. If a particular build is occupying all your agents, we can scope it to only use a certain number of agents (using agent-targeting rules by setting up queues and tags ) and then use the remaining agents to run the other builds.

There is also this other approach where we can use the ‘priority’ attribute to prioritize the job order. Here are more details.

Hope this helps! Let us know for any questions.

Best
Nithya

Hi @nithya,

Thanks for the answer.
The first approach was something in mind, but that would mean we need to keep separate agents for staging and prod. We might need to look on autoscale approach, which spins up new agents when something is in waiting.
Any chance the branch priority feature is in roadmap?

For the second approach, i had question, the priority is set on job order. Is the job referring here to any task in a pipeline? Or the job is a particular build and that is triggered.

Regards,
Suraj

Hi Suraj,

Yeah, we have our AWS Elastic CI Stack that does the auto-scaling. Please give it a go and let us know for any questions.

For Branch Priority, we can use Job Prioritisation. It works across all the builds. But the only caveat is that it can be set only to command steps. We can set the priority dynamically on the steps for the master branch. They would run first even if others on the develop branch have been waiting longer.

For the second approach, i had question, the priority is set on job order. Is the job referring here to any task in a pipeline? Or the job is a particular build and that is triggered.

Each step in a pipeline is run as a job when the build starts.

Hope this helps! Let us know for any questions.

Best,
Nithya

Thanks @nithya for the suggestions.
I will check AWS Elastic CI stack.
We run all steps using commands and calling shell scripts, so may be we can try this priority with adding conditionals on branches in the pipeline.

Is there a way how I can fetch some metrics from buildkite to decide build my autoscaling criteria ?

Hi @surajthakur, you could check out this repo which is used by the elastic stack to perform the scaling. It reads metrics from the Buildkite API to decide whether to scale in or out more instances.