Right now, we look to see if the BUILDKITE_CREATOR_TEAMS environment variable is set. Pretty straight forward.
However, if I were to request a potential solution, I would love to see an integration using a GitHub app. We could install the “Buildkite App” into our organization, and then toggle a setting that says “Block builds from non-organization GitHub users” under the GitHub settings of the pipeline (or someplace similar). Should be easy enough to find organization membership given the information provided by the webhook payload.