Elastic CI Stack for AWS v5.0.0 has been released
Previously experimental Lambda-based scaler is the new default 🚀
Our previously experimental fast autoscaler that scales up much faster is now always used. Instances will automatically scale to demand using the stack defaults and you should see good results without changing any parameters.
Experimental Windows support is now available ![windows|20x20](upload://dGbOAfU1uzuBOkvkoXUbsr5TQNR.png)
A new AMI built on Windows Server 2019 is optionally available to use in the stack for Windows builds. You can enable this by changing the
InstanceOperatingSystem setting from the default
windows. This release includes recent updates to Windows stability on Buildkite agent v3.25.0. Huge thanks to @jeremiahsnapp and @tduffield for their contributions here!
Note: There is a known issue with graceful handling of spot instances under windows. The agent may not disconnect gracefully, and may appear in the Buildkite UI for a few minutes after they terminate (See issue #752). We recommend using Windows on-demand instances for now.
- Support for buildkite/image-builder which can enable you to customize AMIs based off the ones we ship #692 (keithduncan)
- Support for multiple security groups on instances #667 (jdub)
- AMI and Lambda Scaler support more regions: ap-east-1 (Hong Kong), me-south-1 (Bahrain), af-south-1 (Cape Town), eu-south-1 (Milan) #718 (JuanitoFatas)
- Support for loading BuildkiteAgentTokenPath from AWS Parameter Store #601 (jradtilbrook), #625 (jradtilbrook)
- Docker configuration is now isolated per-step #678 (patrobinson) #756 (yob)
- Use EC2 LaunchTemplate instead of a LaunchConfiguration #589 (lox)
- InstanceType default is now
t2.nano) #699 (pda)
- Disable AZRebalancing to prevent running instances being terminated unnecessarily #751
- Bump Buildkite Agent to v3.25.0 #726 (JuanitoFatas)
- Bump Buildkite Agent Scaler to v1.0.2 #724 (JuanitoFatas) 4fafd8e (JuanitoFatas)
- Bump docker to v19.03.13 (linux) and v19.03.12 (windows) and docker-compose to v1.27.4 (linux, windows uses latest choco version) #719 (yob) #723 (JuanitoFatas)
- Bump bundled plugins to the latest versions: secrets, ecr and docker login
And much, much more, we recommend you view the full changelog for this release.
In most cases, you should be able to upgrade in-place older versions of the stack with a CloudFormation stack update using the following template URL:
If you want to launch a new stack, you can use this link (make sure not to use your production AWS account, create a new one for CI):
If you were using them, the following parameters have been removed or reworked:
EnableExperimentalLambdaBasedAutoscalingwas removed (it's the default now)
BuildkiteOrgSlugwas removed – the information reported by buildkite-agent-scaler make it redundant, but consider buildkite-agent-metrics if you need more detailed metric monitoring that supports multiple metric backends
ScaleCooldownPeriodare replaced by the more concise
ScaleInIdlePeriod#586 (jeremiahsnapp) which lets agents self-terminate to scale in when they've been idle after a set period
ScaleDownAdjustmentwere removed - instances will now always try to decrement the ASG desired count when their waiting period for new jobs has elapsed
ScaleUpAdjustmentis replaced by
ScaleOutFactoras the new lambda scaler calculates how many agents are needed at the time.
ScaleOutFactorallows you to multiply the relative quantity provisioned to the ASG, either to slow it down or speed it up