We have 4+ queues being used, and the whole stack is deployed with terraform(using the AMI’s published as part of the Elastic CI stack).
Now, we got a bit stuck in time as the changes to the AMI, scripts to configure the agent have been updated, and our cloudwatch logs streams are taking more and more work to reconfigure. :D
I’m thinking to deploy the cloudformation stack via terraform resource: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudformation_stack
It’ll be used to deploy the stack with several sets of configuration… 1 stack/queue.
Some questions before I start this effort:
is there a supported way to inject our own additional USERDATA? we want to add EFS for the git mirror, install additional security tooling, ssh-keys from our vault, pre download some docker images(as a cache)
is this the cloudformation template: https://github.com/buildkite/elastic-ci-stack-for-aws/blob/main/templates/aws-stack.yml
Any advice/warnings/thoughts appreciated.