Can I assign a CI variable on the parallelism value?

Let’s say I have this step on the Build steps:

steps:
  - label: 'Run tests'
    commands: 'make run_tests'
    parallelism: 4

Can I make the value (e.g. 4) on parallelism as a variable like ${PARALLEL_RUN}?

so it looks like something like this:

env:
  PARALLEL_RUN: 4

steps:
  - label: 'Run tests'
    commands: 'make run_tests'
    parallelism: ${PARALLEL_RUN}

Ultimately, this pipeline is a triggered pipeline from other repositories so I would like to have more flexibility in terms of running the jobs in parallel depending on the codebase’s pipeline running.

Hi @ebanster

No, we cannot use variable interpolation for parallelism and only option here is to use dynamic pipeline where you can use environment variable to define the parallelism for the pipeline definition which will be uploaded using the dynamic pipeline process.

1 Like

Thanks @suma can you show me an example on how this dynamic pipeline can be done?

Hey @ebanster!

You can check our docs that have a complete explanation of how dynamic pipeline works Defining Your Pipeline Steps | Buildkite Documentation, and you can check a working example here GitHub - buildkite/dynamic-pipeline-example: An example of how to generate dynamic build pipelines in Buildkite

Best!

1 Like

@ebanster
you could use YAML anchor/alias instead
here is an example

global_vars:
  - &PARALLEL_RUN
  	4

steps:
  - label: 'Run tests'
    commands: 'make run_tests'
    parallelism: <<: *PARALLEL_RUN
1 Like