Non-agent hook method of disabling the cloning/submodule update into working directory

Features Requests
1

RE: How do I skip the "Preparing Working Directory" step?

Problem: We’re cloning the entire repo and then doing submodule checkout just to get a specific file/folder for CI/CD scripts. The clone/submodule actions performed are outside of our control unless we disable it in the agent’s hooks, but, we are extremely uncomfortable with this solution (see the URL/discussion above). It delays the pipeline execution by several minutes depending on various factors like networking between us and GitHub, the size of the repo, etc.

Solution: Have a top-level boolean key in each YAML block/step which disables the clone/submodule stuff entirely. It needs to still create an empty directory since we, in the commands list in the YAML, do the download of the pipeline.yml or a sparse checkout instead of the clone.

Example of what we do now:

steps:
  - wait

  - label: ":pipeline: Generate Pipeline Steps"
    command:
      - "./.cicd/generate-pipeline.sh > generated-pipeline.yml"
      - "buildkite-agent pipeline upload < generated-pipeline.yml"
      - "buildkite-agent artifact upload generated-pipeline.yml"
    agents:
      queue: "automation-basic-builder-fleet"
    timeout: ${TIMEOUT:-10}

What we’d like to do:

steps:
  - wait

  - label: ":pipeline: Generate Pipeline Steps"
    prep-working-directory: false
    command:
      - "git clone --depth 1 --no-checkout --filter=blob:none $BUILDKITE_REPO cloned-repo"
      - "cd cloned-repo && git checkout $BUILDKITE_COMMIT -- .cicd"
      - "./.cicd/generate-pipeline.sh > generated-pipeline.yml"
      - "buildkite-agent pipeline upload < generated-pipeline.yml"
      - "buildkite-agent artifact upload generated-pipeline.yml"
    agents:
      queue: "automation-basic-builder-fleet"
    timeout: ${TIMEOUT:-10}

The time savings for this should end up being ~1 minute for our pipelines on average, which regardless of how it looks actually ends up being a lot of money and time in a week.

2

Not 100% sure if it meets your use case, but I believe that’s what this plugin is intended to do.

1 Like