Adding per-step custom labels to k8s jobs

We’re evaluating switching to using the k8s agent stack, and it would be very helpful to able to customize the labels applied to the jobs that are submitted into k8s on a per step basis. Specifically I’m hoping to leverage this along with kube-state-metrics to get job durations for specific labels and highlight these in a dashboard. (currently the job labels are just the uuids and optionally agent tags, but that’s too generic). It seems like this could be implemented in a similar way to PodSpecPatch agent-stack-k8s/internal/controller/config/config.go at 8eb53d61131a90232641b38e958bb3836faab615 · buildkite/agent-stack-k8s · GitHub but maybe there’s some other way that I’m not seeing. Thanks!

Hi @nfulwiler ,

Welcome to the Buildkite Support Community! :wave:

Thanks for the feedback. Yes you are right, the current job labels currently cannot be customised. I think our agent stack maintainers would appreciate to hear about this feedback and learn more about your use case. I suggest to raise this feedback here Issues · buildkite/agent-stack-k8s · GitHub.

Cheers!