How to pass variables/data between Commands

Hi guys,

How can I create/modify a variable in a command and this be accessed in a later command? (Using Linux/Bash/Shell scripts)

e.g.

steps:

  • commands: |
    <define/modify VAR here in command or inside .sh file>
  • wait
  • commands: |
    <access VAR here here in command or inside .sh file>

After some reading around subshells etc. it seems like this may be unavoidable. Things I’ve tried:

  • Using ENV variables outside of step level, but can’t modify these
  • Using a VARS.sh as source and trying to modify the var
  • Appending new variables to VARS.sh source and trying to access these in another command

Unless I’m using the wrong syntax there doesn’t seem to be a way to do this, with the BK environment variables being the closest to what I need. Does anyone have any advice?

Thanks for any help
Mitch

EDIT: For anyone else looking, using the meta-data command works!

(Although not sure if it’s the correct solution it works for now)

Hi @Mitch!

Welcome to the community! :blush:

As you mentioned, using meta-data would allow you to store data that works across build steps and build agents. Another alternative is using artifacts; you can upload and download artifacts between steps which will allow you to retain information between steps, i.e folders. There is an example of this here . But meta-data would be the best option for your use-case.

Cheers!

1 Like

Hi Paula, is there a recommended method for sharing secrets between commands? For example, if the first step retrieves a secret and the second step needs to use that secret.

Hi @poltj18

First of all welcome to Buildkite community and thank for your question.

If goal is to share secrets or access secrets for steps in a pipeline then best approach is using secret storage such AWS secrets manager, hashicorp vault etc

If you do not use secrets storage services then next best option would be to use environment hooks so that agent can export the secrets to a step in pipeline.

Please let me know if this helps or if you have any further questions.