🚀 Changes to Environment Variable Precedence


#1

With the introduction of the new YAML step editor in Buildkite, we’ve tweaked the precedence logic of environment variables for those pipelines opting into the beta.

The current precedence/merging logic looks like this:

pipeline_environment. # defined on the pipeline
  merge(step_environment). # defined on the step
  merge(build_environment). # passed through the API or the "New Build" dialog
  merge(buildkite_environment) # $BUILDKITE_ and friends

The fact that custom build environment overrides custom step environment can be problematic. As an example, given a pipeline like this…

env:
  LEVEL: "pipeline"

steps:
  command: "echo $LEVEL"
  env:
    LEVEL: "step"

…if you create a build using the “New Build” dialog and add an environment variable of “LEVEL=build”, it would override all instances of $LEVEL (even if you defined an override at the step level).

We’ve changed it to this:

pipeline_environment.
  merge(build_environment). # 👇 (swapped)
  merge(step_environment). # ☝️ (swapped)
  merge(buildkite_environment)

So creating a build with a custom environment overrides the pipeline environment, but the step environment always takes precedence.

The other notable change is that we’ve stopped the env block in a pipeline.yml file from taking precedence over all other environment variables. Using the same pipeline.yml file from earlier…

env:
  LEVEL: "pipeline"

steps:
  command: "echo $LEVEL"
  env:
    LEVEL: "step"

…if you were to upload this pipeline with buildkite-agent pipeline upload command, the result would be echo pipeline , which is probably not what you’d expect.

We consider this a bug, so it’s been fixed. The result is now echo step as you’d expect. Additionally, any environment variable defined in the top level env block, will be made available to all future buildkite-agent pipeline upload commands.