Hi friends! I’m super new to BK and was wondering why BuildKite has to set up from scratch (e.g. clone the repo, etc) on every step. Is there a way to, for example, share context across steps? I’m just using the BuildKite web UI, and I’m not running the agent directly.
Hi @rohan,
We do this to seamlessly enable the massive parallelism you can get out of Buildkite. Steps don’t presume that any prior state is kept around, which means they don’t need to guarantee they’re running in the same environment.
This requires a bit more consideration upfront, but we find generally pays dividends as peoples’ needs expand!
We do keep around the git working copy when a subsequent build runs on the same agent instance, which should speed up checkouts on long-lived agents, and we do also attempt to optimise how much data we check out each time, but this is limited at times by how Git remote object addressing works under the hood.
For sharing context between steps, you can use artifacts for files and meta-data for shorter strings and the like. We’re hoping to publish more guidance around this at some point in the future, as it’s definitely one of those frequently asked questions for people new to Buildkite!