How to use Secrets & Env Variables?

General
1

Hi – I’m trying to use the Agent secrets to load environment variables for my run.

I’d like to put an entire .env file in an Agent secret, similar to this:

AWS_ACCOUNT_ID=1000050491
AWS_USER_NAME=example-infra
AWS_USER_PASSWORD=VERYLONGSECUREPASSWORD
AWS_ACCESS_KEY_ID=AKIAYMNEXAMPL
AWS_SECRET_ACCESS_KEY=/p/verysecureaccesskey
AWS_POLICY_ARN=arn:aws:iam::1000050491:policy/ACCOUNTInfraPolicy
BUCKET_NAME=SAMPLE_BUCKET_NAME

Then I’d like to use those secrets in my pipeline - so something like this:

env:
  ENV_FILE: $ENVIRONMENT_FILE

steps:
  - name: Lints
    branches: "*buildkite*"
    commands: |
      . source $ENVIRONMENT_FILE
      SHELL=$(which bash) FLOX_DISABLE_METRICS=true flox activate -r bac/bacalhau -t -- pre-commit run --show-diff-on-failure --color=always --all-files
    plugins:
      - docker#v5.11.0:
          image: docker.io/bac/build-container:0.0.32

Can you see anything wrong with this? For example:
– Is this how you load secrets?
– Can you load a secret in the env: field?
– Does each command execute in its own shell (e.g. env variables won’t carry over from line to line)?
– Will Env variables be exposed?

Anything else?

2

Hey @aronchick,

Thank you for reaching out with your questions.

Here is our documentation that explains about how to manage secrets with in the pipelines in a secure way

One of the options is to exporting secrets through agent environment hooks as explained here Managing pipeline secrets | Buildkite Documentation

Regarding your other questions, each step in your pipeline will run on an agent so any environment variables set or available in that command step will all be available. Regarding environment variables getting exposed, yes they will be exposed unless they have a naming convention that matches our redaction rules.

This documentation explains what will get by default redacted Managing log output | Buildkite Documentation and also you have ability to define additional redaction patterns using Buildkite Agent configuration v3 | Buildkite Documentation which you set with agent configuration.

I hope this answers your questions. Please feel free to reach out if you need any further assistance.

Thanks,
Suma

3

thank you very much - i read the whole documentation, but there wasn’t (as far as i could tell) any information about how to set a global ENV variable that could come from Secrets (in the Agent tab) - is that just a variable?

e.g. on Github Actions, one uses ${{ secrets.GOOGLE_APPLICATION_CREDENTIALS }}

anywhere in your pipeline, and it auto replaces with the correct value.