Git Mirrors are not a complete solution to this problem. For some repositories, a bulk of the space is consumed via LFS files, which will not be shared between mirrored repositories.
We’ve actually managed to solve this problem thanks to the configurability of the BUILDKITE_BUILD_CHECKOUT_PATH variable in the global environment hook. This is what we use at Butterfly:
export REPO_SLUG=$(echo "$BUILDKITE_REPO" | cut -d: -f2 | sed 's/\.git$//')
export BUILDKITE_BUILD_CHECKOUT_PATH="$BUILDKITE_BUILD_PATH/$REPO_SLUG"
It works pretty well, but ideally we’d like to have a simple configuration flag for this.
Feature Spec
Add two configuration flags to control checkout path:
checkout_path_includes_pipeline: true by default, settable to false. When true, the pipeline name will be included as part of the checkout path.
checkout_path_includes_hostname: true by default, settable to false. When true, the hostname will be included as part of the checkout path.