Single-step Dry-run from a desktop

I’d like the ability to run a single step in a build pipeline using local changes that are not yet pushed to a remote repository.

We use buildkite to run Hardware-In-the-Loop testing on microcontrollers situated in various labs though-out the world. It would be great to allow developers to use buildkite to test their changes on just one of these test rigs before pushing and submitting a PR.

Thanks for the suggestion and feedback! I think the Buildkite CLI might do the trick for you?

I think this might be close but there are some things missing:

  1. Documentation? I can get some of the commands to work but doing bk build create just ends with exit status 128 and --debug doesn’t provide any meaningful help.
  2. My request was to be able to use a given agent/queue from the cli. It would look something like: bk agent run --step=on-target-s32k-test --artifacts=./build/ where the CLI uploads the binaries under ./build to allow the build step to download them.

This would be helpful for pipelines where building locally was easy but running some of the more complex tests in the pipeline was difficult. Developers could get most of their work done using a local toolchain, getting code written and unit tests running, and then use this feature to validate that they hadn’t regressed any integration tests before they add the noise of a push and PR. Additionally this would reduce the cycle time for debugging these tests since the binaries delivered to the integration step would be built locally and the developer can short-circuit on just one set of tests without waiting for all the previous steps to complete.

Let me give you a made-up example: imagine a company that is building satellites. They might have subsystems broken out into test racks, say reaction wheels, that were used by a build pipeline to verify firmware changes for the vehicle. These rigs might cost upwards of $100K (USD) a piece and may even pose safety risks to be near them. As such there would be only a few and they would not be located near a developer’s desk. The feature I’m requesting would allow a developer in that company to build her binaries at her desk in Zurich and then initiate a test run on this reaction wheel rig in Brussels. Because she has skipped all other steps in the multiple pipelines the company has she’s able to get the test scheduled almost immediately and finds that her change has broken some required functionality within minutes. As we all know, the closer to the source that a software defect is found the less it costs to fix it.

Sorry about the documentation! bk cli is still young, and we could definitely improve things there. @lox is working on improving it.

We’re totally onboard with everything you described, that’s the idea of the bk local run command.

The bk build create command actually creates a build in Buildkite, but the bk local run runs a local agent just like you’ve described. Want to give it a go?

You should be able to run:

bk local run --filter=on-target-s32k-test

I think it uses step labels, and not step id's (https://buildkite.com/docs/pipelines/command-step#command-step-attributes)

I think I’m asking for precisely the opposite of this functionality?

bk build create --filter=on-target-s32k-test

but even if this was supported it wouldn’t work because my on-target tests pull artifacts built in the previous step. What I’m looking for is a way to run the previous step locally:

bk local run --filter=on-target-s32k-test

which totally works (:partyparrot:) but to then use the build artifacts from that local run in a special build in the pipeline. Something that might look like:

bk build create --filter=on-target-s32k-test --upload-artifacts="./build/*.hex"

This would create a special build (i.e. not grouped by the branch name in the list of builds like how PR builds work) that would be given the artifacts specified on the command-line where they match the artifacts property in the step yaml.

It’d also be really cool to find out why bk build create doesn’t work for me but I guess I should open a topic under help for that.

P.S. --filter does use the label which is weird. id would be more appropriate for this.