My opinionated Buildkite Helm chart for fast and efficient docker-compose pipelines

General
1

Hi,

I’ve been working on an opinionated Buildkite Helm chart that’s focused on turn-key docker-compose builds, with heavy emphasis on efficiency and speed. I’ve called it buildkite-agent-swarm for the time being: GitHub - chendo/buildkite-agent-swarm: Buildkite Helm Charts repository

The goals are:

  • Turn-key deployment for docker-compose pipelines
  • Fast and efficient builds
    • Minimise unnecessary overhead from duplicate work (builds, fetching, booting dependencies)
    • Maximising use of compute
      • Spread work across nodes

The key improvements over the now deprecated Helm chart is it uses agent priority to ensure jobs are spread evenly across nodes within a cluster, as well as using a per-node DIND as a Daemonset to minimise the amount of IO done to pull images.

PRs welcome, but don’t expect any support as this is something I’ve put together our own purposes.

2 Likes
2

Hey @chendo!

Thanks for the contribution and welcome: also a welcome here to the Buildkite community :wave:

Definitely looking forward to seeing the continued progress of this / additional PRs / functionality improvement and with the contributions to the repo by the wider community here.

Much obliged!