Agent clusters: pros/cons?

Pipelines
1

We currently use “unclustered” agents, and have 5 agent types that we manage for ~100 pipelines. We are exploring whether to adopt the pipeline “cluster” feature, and I’m trying to better understand the value proposition. Here’s what I’ve gleaned so far:

  1. Clustered agents have finer-grained / better performance metrics
  2. Clusters can be configured to have different security requirements

I’d like to get this group’s thoughts on the pros/cons of this feature. In particular:

  1. Are there other pros/cons of agent clusters I should consider?
  2. What are the common ways of organizing clusters? (The docs mention these, but I’d love to know how y’all are using the feature).
2

Hey @mrstevegross :wave:,

First of all welcome to the Buildkite community and thank you for reaching out to us with your questions :blush:

Here are details on your questions

  1. Are there other pros/cons of agent clusters I should consider?
    Apart from the pros that you listed above, with clusters you will be able to group your agents and pipelines together which gives you better isolation which means only pipelines associated with a specific cluster can have their builds run on the agents in that cluster.

Another advantage is individual users or teams can maintain their own clusters.

There are no particular cons with clusters except that you would need to plan for migration of your pipelines and we have documented the steps here Manage clusters | Buildkite Documentation which helps you in this process

  1. What are the common ways of organizing clusters?
    Common ways to structure your clusters explained in the documentation link that you shared are the main reasons for how you organize them. It is mainly to isolate and group the pipelines and agents so you should use that when you plan to setup clusters.

I hope this helps with your questions. If you have any follow up questions please do let us know.

Thanks,
Suma

3

Cool, thanks for the info. If anyone else on this forum uses clusters, it would be helpful to learn more about your use case; please feel free to reply on-thread. Thanks again!