Hi @MishaLT, I understand that we’re talking about the experience of testing your Buildkite pipeline config. You have the buildkite.yaml stored in your repo, and you want to test changes to it in a branch before merging that to main. But the config has conditionals based on the Git branch that is being built, so you need a way to test those aspects of the config with arbitrary branches, not just the PR branch where you are mocking up the changes. Do I have that right?
There is one thing that I am not sure about - you said:
On pr branch I only can publish RC version. So if I ever made changes to the process of building and publishing stable version, I never can test it, except after merging to main.
So am I understanding correctly that, for the purposes of your testing, it’s OK to run the ‘main’ or ‘release’ conditional of your pipeline against the code in your testing branch? That doesn’t cause problems for your process, and it will be sufficient to validate the pipeline?
If I’m getting this right, then it’s correct that there isn’t a way to e.g. have Buildkite check out a branch named “pipelinetest” but then run the build with variables as though it has checked out the “main” branch. At least, the build.branch variable is always going to be the actual branch that was checked out, and you can’t override that. I understand how this makes testing your changes pretty annoying. I have two suggestions that might make things a bit easier.
First, you could modify your branch conditionals to check a custom variable, rather than build.branch. This variable would be modified while you are testing the different conditionals; otherwise, it is just a copy of build.branch. This would work best if you used a dedicated branch just for testing the pipeline config, and the config had no conditionals that fire on that branch. For example, let’s say your branch is called “pipelinetest” and your actual development branches are “main,” “feature,” and “experiment.” Then your pipeline code might look like this:
env:
# TESTBRANCH: build.branch
TESTBRANCH: "main"
steps:
- command: echo 'This was a build of the main branch'
if: build.env("TESTBRANCH") == 'main'
- command: echo 'This was a build of the staging branch'
if: build.env("TESTBRANCH") == 'staging'
- command: echo 'This was a build of the experiment branch'
if: build.env("TESTBRANCH") == 'experiment'
Now, if you build or commit to your “pipelinetest” branch, a build will run with no steps, so effectively nothing will happen. But if you change the value of TESTBRANCH to “main” and push that commit, it will trigger a build which will exercise the conditionals for “main.” Same for the other two code branches. I’ve tested this out and confirmed that it works. Once you are done testing, you would change your code back to:
env:
TESTBRANCH: build.branch
...
My second suggestion is along similar lines, but instead of editing the pipeline.yaml in your repo and committing it repeatedly, create a new testing pipeline in Buildkite and paste your YAML into the steps editor in the UI. Don’t use the buildkite-agent pipeline upload step. You can target the same git repo where your code lives. But now you could edit the YAML in the Buildkite UI, change the TESTBRANCH variable as appropriate, and run repeated builds to exercise the different conditionals. This way saves you the work of making repeated commits and pushes to your “pipelinetest” branch during this process. Once you are satisfied that the code works the way you want, you can transplant it into your pipeline.yaml file in the repo.
I realize that this still sounds a bit convoluted, but I think these suggestions help address the problem that you’re describing. Please let us know what you think!
welcome to the community!
