Advice on job with different variations

Pipelines
1

Hello,

I want to create a logic that when I start a job from a pipeline to trigger the same steps but with different variables multiple times. For example if a specific variable is added as environment variable then to trigger inside the same job the same steps from an yaml but each of the trigger to be with different environment variables already defined. Pretty much like if environment variable contains PRODUCTION=true then to have the same steps(with small environment variable difference) for stage dev, alpha, staging, with blocking steps for each of those. Else if PRODUCTION is not present or is false then to execute only once the steps based on the environment variable provided without block steps.

What will be the best way to compose this logic? Thanks

2

Hey @UnknownTester404

You should be able to achieve this behaviour using a pipeline like this.

The code is part of a Buildkite pipeline configuration that dynamically generates and uploads additional steps based on the value of an environment variable.

Within the pipeline, it checks if the production environment variable is set to true. If production=true, it dynamically creates and uploads another pipeline step and sets the environment variable to “staging.”

This setup allows us to conditionally add more steps to the pipeline at runtime, depending on the environment variable’s value.


env: 
  production: true
  
steps:
  - label: "Generating Step"
    command: |
      cat <<- YAML | buildkite-agent pipeline upload 
        steps:
          - label: Checking permission
            command: |
              if [ "$$production" == "true" ]; then
                cat <<-YAML | buildkite-agent pipeline upload
                  steps:
                    - label: "Checking permission"
                      command: "echo hi"
                      env: 
                       environment: "staging"
             
              YAML
              fi

Another approach to achieve this is by using the environment hook to set environment variables based on specific conditions. An example of this application can be seen in our documentation.

3

Is there a possibility to have this in the pipeline settings via UI, maybe via bash script which generates the pipeline?

I am thinking on a more holistic matter in terms of if the environment variable is production and not scheduled then create a pipeline, else if production but is cheduled to create another different pipeline. The environment variable ENV will be passed from the UI prompt. If yes, can you help me with some tips on how can I achieve this? Thanks

4

Hey @UnknownTester404 :wave:

You can use bash script to generate this pipeline(Dynamic pipelines) based on the env variable passed with the build via UI.

You have the option to pass env variable when you run new build or when you schedule build via UI

image
image1076×1366 60.3 KB

image
image1792×1308 159 KB

You can read these variables inside your pipeline like $env_var_name and refer the same across your pipeline generation script to conditionally define the logic similar to what Stephanie mentioned.

Hope this helps, let us know if any questions!

Cheers,