Issue: Allow to set required parameters (like in block step) for the build before it starts


#1

https://github.com/buildkite/feedback/issues/488#issuecomment-439412552

Same as here: https://github.com/buildkite/feedback/issues/429

Hi!

Right now NewBuild UI allows to set name, commit, branch, and initial ENV_VARS. This means that there is no way to parametrize build with more human readable options like it is nicely available with https://buildkite.com/docs/pipelines/block-step.

The problem with using build step is that even when you put it at the very top step and start a Build - you need to wait from 5s to sometimes a couple of minutes for the environment to set up (e.g github repo being pulled) until you can pass parameters.

For a fully automated pipeline, you don’t wont to require the user to watch build status closely until it can provide actual info. Especially when the idea is to provide initial input only to limit user context switching.

So acceptance criteria for this would be:

  • Be able to set required parameters for the build before it starts.

Also the color of status for “blocked/paused” build is really similar to green one which could be confusing, but that’s a separate issue.

Am I missing any obvious solution here? (:


#2

@bwplotka thanks for the question! You’re not missing anything here, it’s currently not possible to define block steps with input fields using the UI at the stage.

Having said that…we’re just about to release a new feature in Beta to solve this exact problem. What this space!


#3

We’ve just shipped a new feature in Beta that should hopefully address this for you @bwplotka

https://forum.buildkite.community/t/defining-pipeline-build-steps-with-yaml/79/3

You can now define steps in the Buildkite UI using YAML, giving you access to all the same things that a pipeline.yml file does (in this case, the ability to define block step fields)

Putting a block step at the very beginning will cause the build to start out in a “Blocked” state. You’ll be able to navigate to that build the “unblock” it using the first step - and provide values to the fields defined in the block step.

Let me know how this goes!


#4

Hi, We are already using this feature and that is our mitigation, but it’s not perfect.

The problem is that even when you put this step at a very beginning you still wait some time for the build to set up. What would help here is the same fields options to set for NewBuild component.

Bartek


#5

One thing we haven’t figured out how to do yet (context - YAML steps, dynamically uploaded from source control, and firm guidance to not define steps via the buildkite UI) is have one pipeline trigger another and pass along some parameter values, and have those parameter values be received by the next pipeline’s first block-step and, if it satisfies those, don’t block.


#6

Do you mean you’ve already defined a block step in the Buildkite UI using the new YAML step editor as the first step? Or is the block step still defined in code as the first thing?

If you’re already using the new step editor and there’s still some delay, would you mind shooting us a support email (hello@buildkite.com) with a link to the build so I can try and understand what’s causing the slow down? (or a faster way would be to post some examples in here)

You could solve that with a dynamic pipeline like this…

steps:
  - trigger: "test-pipeline"
    build:
      meta_data:
        some_field: true

…and then inside test-pipeline do this upload.sh | buildkite-agent pipeline upload with something like…

echo "steps:"

if [[ `buildkite-agent meta-data exists some_field -eq "false" ]]; then
echo << EOF
  - block: "Request Release"
    fields:
      - text: "Code Name"
        key: "some_field"
EOF
endif

echo "  - command: 'rest.sh'"

(the above code is untested) but hopefully it conveys the approach!


#7

I’m, so far, uncomfortable with the idea of making pipeline branching logic happen that way - I don’t really want to do that and propose that to my userbase because I predict it might lead to spaghetti.

One of the things I like the most about buildkite is that its pipeline/step definitions are entirely done as data, declarative, with no leak-in of logic within the definitions. I badly want to avoid pipelines-defined-as-code, if at all possible, for as long as possible, behind some kind of break-glass option.

I’m not sure whether that desire’s totally realistic, but that doesn’t make it any less of a desire :)


#8

Do you mean you’ve already defined a block step in the Buildkite UI using the new YAML step editor as the first step?

Yes.

would you mind shooting us a support email (hello@buildkite.com) with a link to the build so I can try and understand what’s causing the slow down?

The slowdown is easy to reason about:

Preparing working directory and bootstrap will always take time. It’s because the pipeline is hooked into github repository that needs to be cloned. So there will be always delay.

We need some block step before the bootstrap (we don’t need that, we just need user input right?) or even better reuse the same NewBuild box that exists. Do you think that might be a good feature request to add? (:


#9

Ah! Sorry for the confusion here, in that image you posted - the block step is still coming from the pipeline upload step. What I meant was create a pipeline the in Buildkite UI that looks like this:

You can define the block step before your pipeline upload call. If you hit “New Build” with a pipeline like this, you’ll get this sort of build:

Where the very first step is a block, and the build starts out in a “blocked” state (as you can see on the right). Once you’ve unblocked the first step, then the rest of the pipeline can continue (and what ever steps your adding via pipeline upload will be added).

Hopefully that makes a bit more sense!


#10

Thanks for your response!

Sorry for the confusion. All makes sense from buildkite standpoint.

The whole delay is because our framework that ensures the pipeline has an initial step for ensuring we can use versioned steps defined in yaml.

How to avoid this framework and have native experience in buildkite that is another issue I think (: So we can close this one.


#11

No problems - I’m glad we were able to get to the bottom of it! :slight_smile: