I’ve just started playing around with GitHub’s Copilot Coding Agent and it seems to be a bit hampered by being unaware of results of its Buildkite jobs. Is there any straightforward way to let the agent retrieve the results of build(s) corresponding to a given PR or SHA?
Hi @ianwremmel,
Are you currently utilising the GitHub Integration with Buildkite? If so, you should see build statuses being reported in PR’s. If not, you should be able to set them up by configuring the integration as described in the documentation above.
As far as I am aware, the Coding Agents are quite limited in scope to GitHub specifically. This means they are unable to perform checks on specific jobs by “opening” the build itself, nor can they imitate this behavior by running API calls to external systems like Buildkite.
As a workaround, you could use tools such as the GitHub CLI (gh), particularly the gh pr comment command, within your Buildkite pipeline steps. For example:
put_comment=$(gh pr comment "$BUILDKITE_BRANCH" --body-file "$LOG_FILE")
This will post a comment to the relevant PR with the contents of your build log or summary, which the Coding Agent could then see for context.
With that being said, you’d need to be sure to handle error codes appropriately, so comments are posted even when a build fails. Also, consider what information to surface in PR comments to avoid exposing sensitive details!
yep, I’ve got the integration installed and I see the check result, but all the check result provides is pass/fail. I was hoping there was some way to tell the agent how to grab job output from buildkite so it could analyze failures and try new things.
I guess maybe I could combine the pre-exit hook with gh to send results back as a comment…
Hi @ianwremmel 
,
Jumping in for Joe here.
Yes, reading up on the conversation, I believe your approach would work. Since pre-exit hooks run after jobs complete (with access to failure details), combining that with gh should allow Copilot see those details 
.
Cheers!