# Native support for Golang plugins **URL:** \ **Category:** Features Requests\ **Created:** [October 29, 2019, 1:51pm UTC](https://forum.buildkite.community/t/native-support-for-golang-plugins/776 "2019-10-29T13:51:56Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![tom](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.buildkite.community/tom/32/30_2.png) [@tom](https://forum.buildkite.community/u/tom)\ **Post date:** [October 29, 2019, 1:51pm UTC](https://forum.buildkite.community/t/native-support-for-golang-plugins/776/1 "2019-10-29T13:51:56Z")
When you’re working in an environment where you’re managing custom plugins that need to work on multiple agent platforms, it can be annoying/difficult to manage hooks. You have to worry about the nuances of how Bash works on the various platforms, or you’re managing duplicate functionality across Bash and Batch files. Also, Bash is a great scripting language, but more and more plugins are used for more complex purposes. Thus, I am proposing first class support for golang hooks. Now, I 100% acknowledge that I can do what I want today by simply writing a Bash script to call my golang binary, but I’m proposing a more native solution where the existing structure of a plugin is replaced with a single go binary. 1. All the hooks in a single build. The exact structure of this can vary, but I can imagine numerous ways to incorporate all the hooks into a single place. 2. Automatic detection that determines which build of the hook we should call. If we establish that you need to build your binaries in a certain way, name them following a certain convention, and put them in a certain place, the buildkite agent could automatically call the correct build based on your platform. 3. A golang library that makes it easy to write these hooks. Similar to how something like cobra is used to write CLI utilities, we could manage a library around hooks. Now, if there is interest in this, I’m willing to offer my (company’s) services to help build/test this out. We are thinking of shifting towards “buildkite plugin driven development” where a lot of our automation is done inside Buildkite pipelines, and the “how” for that automation is stored in plugins. What I’m looking for in terms of a POC is some feature flagged support for this functionality in the native buildkite-agent and ancillary services. What exactly this entails I think can be up for discussion based on the feedback this topic gets. So, to those reading, if you think it’d be useful to be able to write Buildkite Plugins as Go binaries, please vote! ---
**Author:** ![anon18197598](https://avatars.discourse-cdn.com/v4/letter/a/9d8465/32.png) [@anon18197598](https://forum.buildkite.community/u/anon18197598)\ **Post date:** [November 1, 2019, 2:32am UTC](https://forum.buildkite.community/t/native-support-for-golang-plugins/776/2 "2019-11-01T02:32:01Z")
I love this idea! ---
**Author:** ![worxli](https://avatars.discourse-cdn.com/v4/letter/w/51bf81/32.png) [@worxli](https://forum.buildkite.community/u/worxli)\ **Post date:** [October 16, 2020, 6:13pm UTC](https://forum.buildkite.community/t/native-support-for-golang-plugins/776/3 "2020-10-16T18:13:38Z")
Also love this idea, has there been any first steps on this? ---
**Author:** ![pda](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.buildkite.community/pda/32/708_2.png) [@pda](https://forum.buildkite.community/u/pda)\ **Post date:** [October 19, 2020, 12:07am UTC](https://forum.buildkite.community/t/native-support-for-golang-plugins/776/4 "2020-10-19T00:07:08Z")
I don’t believe there’s been any first steps, but there is still interest in it. ---
**Author:** ![moensch](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.buildkite.community/moensch/32/33_2.png) [@moensch](https://forum.buildkite.community/u/moensch)\ **Post date:** [February 17, 2021, 7:59am UTC](https://forum.buildkite.community/t/native-support-for-golang-plugins/776/5 "2021-02-17T07:59:28Z")
I just stumbled upon this and was starting to play with trying to put together some interesting helper libraries to run native Golang plugins with what we got today. There is one hurdle I came across and I wonder if there’s a chance we could get a small change into Buildkite. Today, the `BUILDKITE_PLUGINS` env var holds the full JSON string of the plugins struct. The “currently relevant” plugin parameters are available through `BUILDKITE_PLUGIN__` prefixed variables. Would it be feasible to have the _current plugin_ configuration exported as JSON too the same way the parameters are? In other words: Extend `ConfigurationToEnvironment` with something as simple as: ```auto configJson, err := json.Marshal(p.Configuration) // handle the error envSlice = append(envSlice, fmt.Sprintf("BUILDKITE_PLUGIN_CONFIGURATION_%s=%s", p.Name(), string(configJson)) envSlice = append(envSlice, fmt.Sprintf("BUILDKITE_CURRENT_PLUGIN_NAME=%s", p.Name()) ``` Reference: [agent/plugin.go at fe4464768534f3cd24c4270c3ab2d28fb23f115b · buildkite/agent · GitHub](https://github.com/buildkite/agent/blob/fe4464768534f3cd24c4270c3ab2d28fb23f115b/agent/plugin/plugin.go#L246-L262) Having this would allow a plugin author to easily just: 1. Know what the internal name of the plugin is (this is useful even for bash plugins) 2. Parse the JSON themselves, potentially even into a type stricter than `map[string]interface{}`. I know this wouldn’t give us _native_ Golang plugins the way OP asked for, but I think it would allow us to write plugins in Go super easily. Bootnote: The reason we need the current plugin’s configuration in `BUILDKITE_PLUGIN_CONFIGURATION_` and cannot just rely on parsing the full `BUILDKITE_PLUGINS` json is that within the context of the plugin, I don’t actually know which entry in the plugin list is the appropriate one to look at, especially if the same plugin is called more than once in the same step (which is technically possible today I think). ---
**Author:** ![moensch](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.buildkite.community/moensch/32/33_2.png) [@moensch](https://forum.buildkite.community/u/moensch)\ **Post date:** [February 17, 2021, 8:28am UTC](https://forum.buildkite.community/t/native-support-for-golang-plugins/776/6 "2021-02-17T08:28:59Z")
I hacked something together here: [Export current plugin name and Configuration by moensch · Pull Request #1382 · buildkite/agent · GitHub](https://github.com/buildkite/agent/pull/1382) ---
**Author:** ![james.telfer](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.buildkite.community/james.telfer/32/1063_2.png) [@james.telfer](https://forum.buildkite.community/u/james.telfer)\ **Post date:** [April 6, 2022, 12:42am UTC](https://forum.buildkite.community/t/native-support-for-golang-plugins/776/7 "2022-04-06T00:42:59Z")
In lieu of native support for this, I’ve spiked a plug-in that publishes its binary to a small docker image and bootstraps with a short bash script. It’s currently Alipine based, but could be scratch instead. The overhead of musl isn’t high though. > > > \## Purpose > > Re-implements the plugin in Go, distributed as a binary with Docke…r. The Bash \`post-command\` now executes an appropriately tagged Docker image instead of running AWS CLI commands. > > The Go implementation retains the same error handling and threshold support, and adds a significant amount of information into the annotation, largely removing the need to check out the vulnerabilities separately in the AWS console. > > \### Out of scope > > The tests for Go need to be added to the pipeline; this will be done in a separate PR. > > \### Example output > > Thresholds not exceeded: > \image > > Thresholds exceeded: > \image > > (These examples use obsolete images that purposefully contain vulnerabilities, for testing purposes.) > > \## Context > > This implementation is a proof of concept for using Go in a Buildkite plugin. The use of Docker to distribute is not the only way this idea could be implemented, but is useful when the target agents have no Go toolchain available. There are other ways to do this, with different strengths and weaknesses. Downloading a release binary directly from Github could work too. We don’t have Go on the agents, so `go run` is currently out. In order to discover the plug-in version in use to be able to download it I had to parse the plug-in configuration with `jq`, and accept that if the plug-in is referenced multiple times that only the first version would be used. I’ll have to check whether the current agent version we’re using supports the env variables added from moensch’s PR. It was an enjoyable experience though: it was easier to test, I spent less time sorting through Bash issues, and implementing the annotation rendering template in Go was extremely simple, giving support for proper HTML escaping without any extra effort. Using Go was a lot more verbose than Bash, but being able to use data structures was handy, and the result is pretty simple code (IMO). There are improvements that could be made, but it was a worthwhile spike and we’ll run with it. If there was native support, making the agent more usable as a library instead of shelling out to it would be useful. ---
**Author:** ![paula](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.buildkite.community/paula/32/954_2.png) [@paula](https://forum.buildkite.community/u/paula)\ **Post date:** [April 6, 2022, 4:52am UTC](https://forum.buildkite.community/t/native-support-for-golang-plugins/776/8 "2022-04-06T04:52:39Z")
That is quite a cool concept @james.telfer! 👍 I’ll be sure to let the team know of this approach for if/when we come around to rearchitecting the hooks system ---
**Author:** ![donal](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.buildkite.community/donal/32/1126_2.png) [@donal](https://forum.buildkite.community/u/donal)\ **Post date:** [September 12, 2022, 5:23pm UTC](https://forum.buildkite.community/t/native-support-for-golang-plugins/776/9 "2022-09-12T17:23:50Z")
Has there been any update or progress in this area? We’re mostly interested in step 1 and 2 from the initial suggestion here. We’re deploying bash scripts to wrap all of our binary plugins with a common naming convention, e.g. hooks/checkout-win-amd64 or hooks/checkout-darwin-arm64, and deploying the whole thing. Would you be interested in a PR that standardised some naming here and provided a way to handle: - hooks/checkout/checkout-win-amd64.exe - hooks/checkout/checkout-darwin-arm64 while gracefully preserving the existing behaviour of executing /hooks/checkout.bat and friends? ---
**Author:** ![suma](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.buildkite.community/suma/32/1306_2.png) [@suma](https://forum.buildkite.community/u/suma)\ **Post date:** [September 12, 2022, 7:01pm UTC](https://forum.buildkite.community/t/native-support-for-golang-plugins/776/10 "2022-09-12T19:01:07Z")
Hi @donal First of all welcome to Buildkite community and thank you for reaching out to us with your questions. While it is an area of interest but it is not currently looked at due to other priority items on our roadmap so the situation is still the same on step 1 and 2. Now regarding PR that standardize naming convention, we definitely welcome your contributions and our team would be happy to review the PR.