Can I `include` one pipeline yaml in another?

Features Requests
1

I’d like to sort of include one pipeline in another one.
Currently I can add a step to the first pipeline forcing it to upload my second pipeline. eg
buildkite-agent pipeline upload mysecond-pipeline.yml
However with this I’d have to duplicate all definitions in the second yaml file as well
I want to avoid that and create more modular pipelines. Can this be done?

2

Hi @js-cb

Please can you explain in more detail what you mean by “I have to duplicate all definitions in the second yaml” ?

You can have first pipeline where one of the step is pipeline upload then have remaining steps in your first pipeline. Only the steps you want to be part of second pipeline you can include in pipeline definition that you are uploading.

Is that you need to run same steps in both pipelines ?

3

See this example. I tend to prefer cat over echo as it reads cleaner.

In your pipeline.yml file you need a Pipeline upload step:

steps:
  - label: "Yada"
    command: ".buildkite/pipeline.sh | buildkite-agent pipeline upload"

Then in your pipeline.sh file have something like this:

#!/bin/bash
set -eu

cat << EOF
steps:
  - label: "Tada"
    command: "npm start"
EOF
4

Example:

_agents: &agents
  queue: docker

I call that a definition.
And I use it here:

steps:

- label: 'Cache checkout and set metadata'
  agents: *agents

If I had the ability to include yaml files then I would not have to repeat these in all YAML files which i upload using the command I mentioned

5

Hey @js-cb!

The alternative that iammcgaber offered it’s a good one, and it’s what we recommend for these cases.

Best!