Dynamic number of agents per instance

Hi,

In our current stack configuration we have a fixed number for the AgentsPerInstance parameter. I was wondering if there’s a way to scale that depending on the instance type. For example, a 12xlarge could handle 24 agents but a 24xlarge could support 48.
We’re currently struggling with spot instance availability and being more flexible with our allowed instance types could help.

Thanks!

Hey Abraham :wave:

Welcome back to the Buildkite Forums!

There isn’t a option that would allow this in the CloudFormation stack template, but you could easily do this with a bootstrap script that edits the agent config!

The template has a BootstrapScriptUrl parameter which will run before the agent starts, that would let you perform any logic needed to alter the spawn count.

For example, you could get the instance type using the ec2 metadata ec2-metadata --instance-type and depending on the value returned you could then alter the spawn value that’s set in /etc/buildkite-agent/buildkite-agent.cfg.

If you’d prefer to not use the config file there is also a env which controls this; BUILDKITE_AGENT_SPAWN you can read more about it here.

Hope that helps!

Hi,

Thanks for your suggestions. I was able to control the number of agents by modifying the .cfg as you suggested (although my first attempt with simply calling export BUILDKITE_AGENT_SPAWN=N didn’t appear to have any effect).