Are there plans to offer functionality that allows us to control whether or not builds can be triggered by pull requests that are from “untrusted users?”
Here is our scenario. We are using Buildkite as a replacement for TravisCI/AppVeyor to run pull request verification tests. The test definitions are kept in a YAML file that is dynamically loaded using the buildkite-agent. We’re concerned that a malicious user could open up a pull request and modify the YAML in a malicious manner.
We’re hoping there could be a way we could “quarantine” those builds until the pull request could be reviewed by a trusted user.
Im aware of that setting, but that doesnt meet my requirments. I need to allow builds to be (or not to be) triggered based on the github user, regardless of whether or not they are on a fork.
Unfortunately this functionality isn’t something that buildkite does just yet, though this use-case is certainly on our radar and something that we’re planning on accounting for in future!
As a potential work around for now I’d suggest a hard coded (via the buildkite UI) a step that runs in these public pipelines that can sanity check the origin on the code, and can then prepend a block step before any uploaded dynamic pipeline steps are added (potentially also shooting a notification off to your team to let them know that the pipeline needs review). Does this sound like it’d do the job?
We’re keen to remove this extra legwork in the future once we get a better idea of all the potential use-cases and approaches that people need to support for running their public builds/pipelines.
@justin thanks for getting back to me. Your workaround is actually the implementation we came up with, and it opened up a ton of options for other really interesting work. For example, we were able to build our own DSL on top of your Pipeline DSL. So, in terms of priority, my original ask has gone away, but I still think a native way to authorize public users would be a useful product feature.
@tom Awesome — that’s great to hear! We’d love to hear the details of your implementation as we’re actively thinking about solving things in this space…a native way of managing this is absolutely on the cards.
Right now, we look to see if the BUILDKITE_CREATOR_TEAMS environment variable is set. Pretty straight forward.
However, if I were to request a potential solution, I would love to see an integration using a GitHub app. We could install the “Buildkite App” into our organization, and then toggle a setting that says “Block builds from non-organization GitHub users” under the GitHub settings of the pipeline (or someplace similar). Should be easy enough to find organization membership given the information provided by the webhook payload.
Looked into this for our own integration, and I’m concerned about using BUILDKITE_CREATOR_TEAMS for authorization purposes. For a webhook-triggered build, the creator seems to be inferred based on the Git commit details of the triggered commit? And those are easily forgeable.
What we’re doing instead is detecting the branch prefix (username:branch) for third-party forks, and validating that username against our GitHub org membership.
[[ "$BUILDKITE_SOURCE" = webhook ]]; then
echo "Build triggered by webhook; checking whether branch is trusted"
if [[ "$BUILDKITE_BRANCH" = *:* ]]; then
# The GitHub username is everything up to the first `:` character. This
# code is careful to parse something like `malicious_user:benesch:benesch`
# as `malicious_user`, because `benesch:benesch` is the branch name
# under the user's control and can contain colons too.
github_user=${BUILDKITE_BRANCH%%:*}
echo "Branch ${BUILDKITE_BRANCH} detected as third-party"
echo "Checking whether GitHub user ${github_user} is a member of the build-authorized team"
# NOTE(benesch): the --fail option below is what makes this actually
# abort the build if the user is not a member of the team!
curl --fail -H 'Authorization: Bearer '{{ github_token | shell_escape }} "https://api.github.com/orgs/<ORG>/teams/<TEAM>/memberships/$github_user"
else
echo "Branch ${BUILDKITE_BRANCH} does not appear to be a third-party branch; skipping security check"
fi
else
echo "Build triggered by a trusted source ($BUILDKITE_SOURCE); skipping security check"
fi
You can think of that bit as being set by Terraform before we install the hook on the builder instance. (The reality is more complicated, but that’s the idea.)