Public Build Security on PRs

Beta Support & Feedback Public Builds
#1

Are there plans to offer functionality that allows us to control whether or not builds can be triggered by pull requests that are from “untrusted users?”

Here is our scenario. We are using Buildkite as a replacement for TravisCI/AppVeyor to run pull request verification tests. The test definitions are kept in a YAML file that is dynamically loaded using the buildkite-agent. We’re concerned that a malicious user could open up a pull request and modify the YAML in a malicious manner.

We’re hoping there could be a way we could “quarantine” those builds until the pull request could be reviewed by a trusted user.

#2

There is already an option the pipeline settings to not start builds when the PR comes from a fork:

Build pull requests from third-party forked repositories . Creates builds for pull requests from third-party forks.

You could just leave that disabled, so you wouldn’t automatically kick off builds for third-party changes to your repo.

#3

Im aware of that setting, but that doesnt meet my requirments. I need to allow builds to be (or not to be) triggered based on the github user, regardless of whether or not they are on a fork.

#4

Hey there @tom!

Unfortunately this functionality isn’t something that buildkite does just yet, though this use-case is certainly on our radar and something that we’re planning on accounting for in future!

As a potential work around for now I’d suggest a hard coded (via the buildkite UI) a step that runs in these public pipelines that can sanity check the origin on the code, and can then prepend a block step before any uploaded dynamic pipeline steps are added (potentially also shooting a notification off to your team to let them know that the pipeline needs review). Does this sound like it’d do the job?

We’re keen to remove this extra legwork in the future once we get a better idea of all the potential use-cases and approaches that people need to support for running their public builds/pipelines.

Cheers,

Justin.

#5

@justin thanks for getting back to me. Your workaround is actually the implementation we came up with, and it opened up a ton of options for other really interesting work. For example, we were able to build our own DSL on top of your Pipeline DSL. So, in terms of priority, my original ask has gone away, but I still think a native way to authorize public users would be a useful product feature.

1 Like
#6

@tom Awesome — that’s great to hear! We’d love to hear the details of your implementation as we’re actively thinking about solving things in this space…a native way of managing this is absolutely on the cards.

#7

Right now, we look to see if the BUILDKITE_CREATOR_TEAMS environment variable is set. Pretty straight forward.

However, if I were to request a potential solution, I would love to see an integration using a GitHub app. We could install the “Buildkite App” into our organization, and then toggle a setting that says “Block builds from non-organization GitHub users” under the GitHub settings of the pipeline (or someplace similar). Should be easy enough to find organization membership given the information provided by the webhook payload.

#8

Looked into this for our own integration, and I’m concerned about using BUILDKITE_CREATOR_TEAMS for authorization purposes. For a webhook-triggered build, the creator seems to be inferred based on the Git commit details of the triggered commit? And those are easily forgeable.

What we’re doing instead is detecting the branch prefix (username:branch) for third-party forks, and validating that username against our GitHub org membership.