When running a build (based on an existing build), re-use environment

Features Requests
1

When developing a pipeline, I often need to iterate a bunch on it. I have a git branch for my work. I trigger the pipeline, configuring (1) the branch and (2) the environment. After the build finishes, I make some changes, and retrigger the pipeline (with the “new build” button on the build page).

However: the dialog for the new build reuses my branch but not my custom environment. This is a little annoying; I have to retype the environment for my test. As long as “new build” re-uses the branch setting, why not the environment as well?

2

Hi @mrstevegross! :wave: thank you for raising this.

While the “new build” feature conveniently reuses the branch setting, it doesn’t carry over the environment configuration. This requires manual re-entry, which can be a little inconvenient when trying to iterate on a pipeline.
You can get around this partially by hardcoding those environment variables into your pipeline YAML, though you would still need pass any alternate values through the environment of the Rebuild dialog.

Here is an approach we can take:

env:
 MY_VAR1: "foo"
 MY_VAR2: "bar"
steps:
 - label: "My Step"
  command: |
   echo "My environment variable: $MY_VAR1"
3

Right, this is why I’m submitting the feature request :) Please consider making the environment auto-repopulate with the environment from the previous build.

4

Hey @mrstevegross :wave:

Taking a look through the conversation here, using build dialog box would be good when you want to pass environment variable which is different or never existed before. But for cases where we need to use same environment variables from previous build, we would recommend having that as part of the pipeline code as env variable as Amna mentioned before. In your pipeline script you can validate if the values for env variable are present, if not set values as needed via the pipeline script.

Cheers,
Priya

5

That’s not quite the use case I’m dealing with. In my case, I have a pipeline that can run in dry-run mode (specified via environment) for test purposes. When developing/testing, I want to repeatedly run it in dry-run mode, and don’t want to have to manually re-enter the environment each time.

In production, the environment is not specified, however.

6

Hey @mrstevegross :wave:

Thanks for clarifying! In that case i’m wondering, if you can set the mode in your pipeline code as environment variable, see if its available in the environment(in case of production this will be available) if not you can set “dry-run” to be the mode by default. So the idea is to have a default in the pipeline code and override it only when needed incase of production. Will that help?

Cheers,
Priya

7

Hmmm, not really… I just want to be able to iterate quickly, and that means not re-entering settings I’ve already entered once. Since “New Build” automatically re-uses the branch, why not the environment as well?

8

Hi @mrstevegross !

Unfortunately, Buildkite currently doesn’t have functionality present in the UI to carry over environment variables from previous builds. However, the functionality you are referring to is very easily accomplished via both our REST API and with our own bk CLI tool.

Here is a REST API example:

curl -H "Authorization: Bearer $TOKEN" \
  -X POST "https://api.buildkite.com/v2/organizations/{org.slug}/pipelines/{pipeline.slug}/builds" \
  -H "Content-Type: application/json" \
  -d '{
    "commit": "abcd0b72a1e580e90712cdd9eb26d3f1cd09c8",
    "branch": "main",
    "message": "Testing all the things :rocket:",
    "author": {
      "name": "Your Name",
      "email": "name@email.com"
    },
    "env": {
      "MY_ENV_VAR": "some_value"
    },
    "meta_data": {
      "some build data": "value",
      "other build data": true
    }

Here’s a document for your reference: Builds API | Buildkite Documentation

And here’s an example for CLI:

bk build new --pipeline my-new-pipeline --branch main --env foo=bar --env hello=world

Thanks a lot!

9

Sure, the CLI is a workable solution for this. Thanks!