Pipeline updates through API don't update env

Updating a yaml pipeline through the API doesn’t allow to update the “env” of the pipeline. Here:

this curl -X PATCH should update the “env” of the pipeline configuration to the current commit, but it doesn’t. It is still “HEAD” after the call, both in the configuration field, as well as in the steps. The top-level env seems effectively ignored.

curl -H "Authorization: Bearer ${CRUISE_PIPELINE_TOKEN}"         -X PATCH         "https://api.buildkite.com/v2/organizations/cruise/pipelines/${CRUISE_PIPELINE}"         -d '{
          "env": { 
            "CRUISE_PIPELINE_REVISION": "'"${BUILDKITE_COMMIT}"'"
          }
        }'
{
  "id": "407b8fb0-d830-4c6d-8da2-b850b5f3c839",
  "url": "https://api.buildkite.com/v2/organizations/cruise/pipelines/cruise-dev-ep8785",
  "web_url": "https://buildkite.com/cruise/cruise-dev-ep8785",
  "name": "cruise-dev-ep8785",
  "description": "Testing for [EP-8785]",
  "slug": "cruise-dev-ep8785",
  "repository": "REDACTED",
  "branch_configuration": null,
  "default_branch": "develop-stable",
  "skip_queued_branch_builds": false,
  "skip_queued_branch_builds_filter": null,
  "cancel_running_branch_builds": false,
  "cancel_running_branch_builds_filter": null,
  "provider": {
    "id": "github_enterprise",
    "settings": {
      "trigger_mode": "code",
      "build_pull_requests": true,
      "pull_request_branch_filter_enabled": false,
      "skip_pull_request_builds_for_existing_commits": true,
      "build_pull_request_forks": false,
      "prefix_pull_request_fork_branch_names": true,
      "build_tags": false,
      "publish_commit_status": true,
      "publish_commit_status_per_step": false,
      "separate_pull_request_statuses": false,
      "publish_blocked_as_pending": false,
      "repository": "cruise/cruise"
    },
    "webhook_url": "https://webhook.buildkite.com/deliver/26c68136fbb360e69e20a695bfa4e3ad2f51320fc34cc3ffee"
  },
  "builds_url": "https://api.buildkite.com/v2/organizations/cruise/pipelines/cruise-dev-ep8785/builds",
  "badge_url": "https://badge.buildkite.com/98fa1f13b8fadbeb3baaee99171871cf0195a92cdb96c09ba2.svg",
  "created_at": "2019-04-19T15:55:56.854Z",
  "env": {
    "CRUISE_PIPELINE_REVISION": "115a02980b5ccf3e6f6e4b8c80a537c61a9e59ef"
  },
  "scheduled_builds_count": 0,
  "running_builds_count": 1,
  "scheduled_jobs_count": 4,
  "running_jobs_count": 3,
  "waiting_jobs_count": 0,
  "configuration": " env:\n   CRUISE_PIPELINE_REVISION: \"HEAD\"\n   \n steps:\n  - command: >-\n      git show $${PIPELINE_GIT_REVISION}:.buildkite/cruise.yml |\n        buildkite-agent pipeline upload\n    label: \":pipeline:\"\n    agents:\n    - \"size=tiny\"\n    env:\n      NO_LFS: 1",
  "steps": [
    {
      "type": "script",
      "name": ":pipeline:",
      "command": "git show $${PIPELINE_GIT_REVISION}:.buildkite/cruise.yml |\n  buildkite-agent pipeline upload",
      "artifact_paths": null,
      "branch_configuration": null,
      "env": {
        "NO_LFS": "1",
        "CRUISE_PIPELINE_REVISION": "HEAD"
      },
      "timeout_in_minutes": null,
      "agent_query_rules": [
        "size=tiny"
      ],
      "concurrency": null,
      "parallelism": null
    }
  ]
}

Oh, it looks like you have two ENV’s happening now… one at the pipeline level, and the other inside the configuration property. Sorry about that.

Can you PATCH the entire configuration property to define a new pipeline YAML with the new env values inside of it? I believe that’s what we were intending, but we’ve missed a validation that should make the API return a friendly error to tell you that you can no longer update env directly.

Alright. Haven’t tried this, but I assume it’s the same for steps? Like so:

env, steps: Deprecated for YAML pipelines, may contains inconsistent data
configuration: The whole configuration as a single YAML string, this is the only thing that matters.

Mhh, I wrote a bit of python code to actually generate these yaml-in-json requests. But looks like patching the configuration property isn’t working either.

Here is the relevant snippet of python:

136 def upload_pipeline(config):                                                                                                                                                                             
137     logging.info("Uploading config: %s", config)                                                                                                                                                         
138     data = json.dumps({"configuration": config})                                                                                                                                                         
139                                                                                                                                                                                                          
140     res = requests.patch(data=data, **(bk_requests.pipeline_params()))                                                                                                                                   
141     logging.debug("Request URL: %s", res.request.url)                                                                                                                                                    
142     logging.debug("Request Body: %s", res.request.body)                                                                                                                                                  
143                                                                                                                                                                                                          
144     logging.debug("Response:\n%s", res.text) 

… and the logging output. The request URL and Body look as I would expect them, and I am getting a 200, but the response and the UI still have “HEAD” and not “FOO”:

$ ci/buildkite/upload_pipeline.py --pipeline=cruise-dev-ep8785 --template_param REVISION=foo -v=1 .buildkite/cruise_start.yaml
I0422 14:40:36.430133 140667532629824 upload_pipeline.py:96] Reading pipeline config from .buildkite/cruise_start.yaml
I0422 14:40:36.431143 140667532629824 upload_pipeline.py:152] New config:
env:
   CRUISE_PIPELINE_REVISION: foo

steps:
  - command: >-
      git show $${PIPELINE_GIT_REVISION}:.buildkite/cruise.yml |
        buildkite-agent pipeline upload
    label: ":pipeline:"
    agents:
    - "size=tiny"
    env:
      NO_LFS: 1
I0422 14:40:36.432090 140667532629824 connectionpool.py:813] Starting new HTTPS connection (1): api.buildkite.com:443
I0422 14:40:36.866827 140667532629824 connectionpool.py:393] https://api.buildkite.com:443 "GET /v2/organizations/cruise/pipelines/cruise-dev-ep8785 HTTP/1.1" 200 None
About to apply diff to cruise/cruise-dev-ep8785
--- 

+++ 

@@ -1,5 +1,5 @@

 env:
-   CRUISE_PIPELINE_REVISION: "HEAD"
+   CRUISE_PIPELINE_REVISION: foo
 
 steps:
   - command: >-
Apply changes these changes to cruise-dev-ep8785 [y/N]: y
I0422 14:40:38.033922 140667532629824 upload_pipeline.py:137] Uploading config: env:
   CRUISE_PIPELINE_REVISION: foo

steps:
  - command: >-
      git show $${PIPELINE_GIT_REVISION}:.buildkite/cruise.yml |
        buildkite-agent pipeline upload
    label: ":pipeline:"
    agents:
    - "size=tiny"
    env:
      NO_LFS: 1
I0422 14:40:38.034631 140667532629824 connectionpool.py:813] Starting new HTTPS connection (1): api.buildkite.com:443
I0422 14:40:38.476830 140667532629824 connectionpool.py:393] https://api.buildkite.com:443 "PATCH /v2/organizations/cruise/pipelines/cruise-dev-ep8785 HTTP/1.1" 200 None
I0422 14:40:38.477874 140667532629824 upload_pipeline.py:141] Request URL: https://api.buildkite.com/v2/organizations/cruise/pipelines/cruise-dev-ep8785
I0422 14:40:38.477984 140667532629824 upload_pipeline.py:142] Request Body: {"configuration": "env:\n   CRUISE_PIPELINE_REVISION: foo\n\nsteps:\n  - command: >-\n      git show $${PIPELINE_GIT_REVISION}:.buildkite/cruise.yml |\n        buildkite-agent pipeline upload\n    label: \":pipeline:\"\n    agents:\n    - \"size=tiny\"\n    env:\n      NO_LFS: 1"}
I0422 14:40:38.478218 140667532629824 upload_pipeline.py:144] Response:
{
  "id": "407b8fb0-d830-4c6d-8da2-b850b5f3c839",
  "url": "https://api.buildkite.com/v2/organizations/cruise/pipelines/cruise-dev-ep8785",
  "web_url": "https://buildkite.com/cruise/cruise-dev-ep8785",
  "name": "cruise-dev-ep8785",
  "description": "Testing for [EP-8785]",
  "slug": "cruise-dev-ep8785",
  "repository": "SOMEREPO",
  "branch_configuration": null,
  "default_branch": "develop-stable",
  "skip_queued_branch_builds": false,
  "skip_queued_branch_builds_filter": null,
  "cancel_running_branch_builds": false,
  "cancel_running_branch_builds_filter": null,
  "provider": {
    "id": "github_enterprise",
    "settings": {
      "trigger_mode": "code",
      "build_pull_requests": true,
      "pull_request_branch_filter_enabled": false,
      "skip_pull_request_builds_for_existing_commits": true,
      "build_pull_request_forks": false,
      "prefix_pull_request_fork_branch_names": true,
      "build_tags": false,
      "publish_commit_status": true,
      "publish_commit_status_per_step": false,
      "separate_pull_request_statuses": false,
      "publish_blocked_as_pending": false,
      "repository": "cruise/cruise"
    },
    "webhook_url": "https://webhook.buildkite.com/deliver/SOMEHOOK"
  },
  "builds_url": "https://api.buildkite.com/v2/organizations/cruise/pipelines/cruise-dev-ep8785/builds",
  "badge_url": "https://badge.buildkite.com/SOMEBADGE.svg",
  "created_at": "2019-04-19T15:55:56.854Z",
  "env": {
    "CRUISE_PIPELINE_REVISION": "115a02980b5ccf3e6f6e4b8c80a537c61a9e59ef"
  },
  "scheduled_builds_count": 0,
  "running_builds_count": 0,
  "scheduled_jobs_count": 3,
  "running_jobs_count": 1,
  "waiting_jobs_count": 0,
  "configuration": "env:\n   CRUISE_PIPELINE_REVISION: \"HEAD\"\n\nsteps:\n  - command: >-\n      git show $${PIPELINE_GIT_REVISION}:.buildkite/cruise.yml |\n        buildkite-agent pipeline upload\n    label: \":pipeline:\"\n    agents:\n    - \"size=tiny\"\n    env:\n      NO_LFS: 1",
  "steps": [
    {
      "type": "script",
      "name": ":pipeline:",
      "command": "git show $${PIPELINE_GIT_REVISION}:.buildkite/cruise.yml |\n  buildkite-agent pipeline upload",
      "artifact_paths": null,
      "branch_configuration": null,
      "env": {
        "NO_LFS": "1",
        "CRUISE_PIPELINE_REVISION": "HEAD"
      },
      "timeout_in_minutes": null,
      "agent_query_rules": [
        "size=tiny"
      ],
      "concurrency": null,
      "parallelism": null
    }
  ]
}

@haraldschioberg :wave: howdy! I found a bug this morning that meant the env property wasn’t being updated after it was changed by patching configuration.

Can you try again and let me know? We still have the issue where env can’t be changed directly by the POST (you need to go via the YAML configuration) - but hopefully this helps.

I have a similar problem. Seems REST API for PATCHing steps and configuration does nothing (yet reports HTTP OK).

I found (see above) then when you are using yaml pipelines then you must patch (aka replace) the configuration property, which is the whole thing as a single yaml string. Patching either env or steps is a no-op.

I think I’m having the same problem as @ericberic; using the REST API to PATCH configuration has no effect. For example:

curl -X PATCH https://api.buildkite.com/v2/organizations/{organization}/pipelines/{pipeline} -H "Authorization: Bearer {token}" -d '{"configuration": "steps: - command: echo \"hello world\""}'

does not change the configuration of the pipeline; the response body has the exact same value for configuration as before.

Any progress or update on this? Was trying to update a pipeline steps via the API and not seeing any change either.

Hey! I’ve just released an update that makes the REST API accept the configuration parameter for POST and PATCH requests. Sending this parameter in your requests will change any UI Steps pipelines to the YAML Steps pipelines beta :raised_hands:

The env parameter can be set in the configuration YAML.

Please let me know if you hit any issues :blush:

I still can’t get this working with the recommendations above. Can you post an example for updating the yaml steps of a pipeline please.

@robbalmbra have you asked support@buildkite.com? If you send us more details we can help you there.