Env var across steps

Pipelines
1

Hi there,

We are currently working on this pipeline where we would use the first plugin to generate a GITHUB_TOKEN as environment variable, which we then would use it in the following step to fetch one of our org’s own plugin repo.

The high-level code looks like this

steps:
  - label: ":key: Create GitHub App token"
    key: "create-github-app-token"
    plugins:
...
      - ./.buildkite/create-github-app-token:
          env-var-name: GITHUB_TOKEN

  - label: ":slack: Register fail notify"
    key: "notify-setup"
    plugins:
...
      - "https://x-access-token:${GITHUB_TOKEN}@github.com/xxx/xxx.git#main": ~

However, I did find the env var seems not carried over cross different steps. Therefore, we considered generate dynamic pipeline at the first step for the GitHub token plugin like this:

steps:
  - label: ":key: Create GitHub App token"
    key: "create-github-app-token"
    plugins:
...
      - ./.buildkite/create-github-app-token:
          env-var-name: GITHUB_TOKEN
    command: |
          /bin/bash .buildkite/scripts/generate_pipeline.sh

which in the script it will be able to access the GITHUB_TOKEN in the current step and build following steps.

Just wondering if this is the only approach if we wanna use generated env var across different steps? Is there a better way we can adopt to make this possible?

2

Hi @byao1031 ,

Welcome to the Buildkite community!

Based on your use case using different plugins and env variable referenced in the plugins URL, your approach in generating a dynamic pipeline is the best option. There are also other options that you can consider such as

You can also checkout/clone your second repo via the command (rather than using plugins). For example:

  - label: ":slack: Register fail notify"
    key: "notify-setup"
    command: |
      set +x
      
      # Clone YOUR repo
      git clone https://x-access-token:$${GITHUB_TOKEN}@github.com/xxx/xxx.git
      
      echo "✅ Successfully cloned repo"

Hope that helps!

3

Hi Dahtey,

I want to confirm if we can use buildkite_metadata to carry the env var over steps? Also how would we use ssh key to connect Github repository and Buildkite agent just wondering?

Thanks

4

Hi @byao021031 I don’t suggest using meta-data as it is a publicly visible variable across builds, the better option would be to set the GITHUB_TOKEN as a secret as mentioned previosly and access it from the next step.
For using ssh key to connect Github repository and the agent, different platforms show a more specific process on where to generate the SSH key to use with your agent. After the key is generated and stored in the agent, the same SSH key will need to be added to the user’s setting in Github. This document outlines the process and methods to do this on Github
More details on how to create the ssh key for the agent and Github along with other configurations can be found here.