Running gradlew command from docker plugin does not work

Docker
1

Hello,

I have a java application in gradle. I have managed to setup an agent running on Linux (with docker socket volume attached) and the pipeline has a simple step of uploading the pipeline.yml file.

In my project, this is my pipeline.yml

steps:
  - label: ":gradle: Build Jar"
    command: sh gradlew clean build
    plugins:
      - docker#v5.9.0:
          image: "gradle:jdk21"
          propagate-environment: true
          volumes:
            - "/var/run/docker.sock:/var/run/docker.sock"

However when it runs, it always says that gradlew file could not be found. I’ve validated that it does exist when buildkite clones the repo, so I am really not sure what else it could be. I’ve tried the example on the docs of using docker-compose, and that does not work either.

$ docker run -t -i --rm --init --volume /buildkite/builds/694a35295247-1/myapp/myapp-api:/workdir --volume /var/run/docker.sock:/var/run/docker.sock --volume /buildkite/builds/694a35295247-1/myapp/myapp-api:/home/gradle/src --workdir /workdir --env BUILDKITE_BUILD_ID --env BUILDKITE_PIPELINE_SLUG --env BUILDKITE_REBUILT_FROM_BUILD_ID --env BUILDKITE_TRIGGERED_FROM_BUILD_NUMBER --env BUILDKITE_REPO_SSH_HOST --env BUILDKITE_BUILD_URL --env BUILDKITE_MESSAGE --env BUILDKITE_PROJECT_PROVIDER --env BUILDKITE_BUILD_AUTHOR_EMAIL --env BUILDKITE_PIPELINE_NAME --env BUILDKITE_PIPELINE_PROVIDER --env BUILDKITE_SCRIPT_PATH --env BUILDKITE_AGENT_ID --env BUILDKITE_TAG --env BUILDKITE_BRANCH --env BUILDKITE_LABEL --env BUILDKITE_COMMAND --env BUILDKITE_COMMIT --env BUILDKITE_ARTIFACT_PATHS --env BUILDKITE_BUILD_NUMBER --env BUILDKITE_PULL_REQUEST_REPO --env BUILDKITE_RETRY_COUNT --env BUILDKITE_BUILD_CREATOR_EMAIL --env BUILDKITE_PULL_REQUEST --env BUILDKITE_PIPELINE_ID --env BUILDKITE_REBUILT_FROM_BUILD_NUMBER --env BUILDKITE_STEP_ID --env BUILDKITE_ORGANIZATION_SLUG --env BUILDKITE_SOURCE --env BUILDKITE_BUILD_AUTHOR --env BUILDKITE_PROJECT_SLUG --env BUILDKITE_REPO --env BUILDKITE_JOB_ID --env CI --env BUILDKITE_AGENT_META_DATA_QUEUE --env BUILDKITE_PLUGINS --env BUILDKITE_PULL_REQUEST_BASE_BRANCH --env BUILDKITE_AGENT_NAME --env BUILDKITE_STEP_KEY --env BUILDKITE_BUILD_CREATOR --env BUILDKITE_TRIGGERED_FROM_BUILD_PIPELINE_SLUG --env BUILDKITE_TIMEOUT --env BUILDKITE_TRIGGERED_FROM_BUILD_ID --env BUILDKITE_PIPELINE_DEFAULT_BRANCH --env BUILDKITE --label com.buildkite.job-id= --label com.buildkite.pipeline_name= --label com.buildkite.pipeline_slug= --label com.buildkite.build_number=34 --label com.buildkite.job_id= --label com.buildkite.job_label=:gradle:\ Build\ Jar --label com.buildkite.step_key= --label com.buildkite.agent_name= --label com.buildkite.agent_id= gradle:jdk21-alpine /bin/sh -e -c sh\ gradlew\ clean
sh: can’t open ‘gradlew’: No such file or directory

2

Hey @ldev654 :wave:

Hope you’re well and welcome to the Buildkite Forum!

What’s happening here is that by default; images that are run by the Docker plugin don’t automatically have the current Buildkite build directory mounted (by default, the Docker plugin works in the /workdir directory within the spun up container, this can be changed however).

Assuming that this is a standard Gradle application with gradlew script within the root directory within the cloned repository on the build host: mounting it like so should suffice with a clean build/other Gradle action:

steps:
  - label: ":gradle: Build Jar"
    command: sh ./gradlew clean build
    plugins:
      - docker#v5.9.0:
          image: "gradle:jdk21"
          propagate-environment: true
          volumes:
            - "${pwd}:/workdir"

Hope that helps :slight_smile:

3

Hello @james.s,

Thank you for replying fast and with the information.

I am still seeing the issue however. First I tried and I get docker: invalid spec: :/workdir: empty section between colons., so I changed ${pwd} to ${PWD} which made that go away. But, I am still seeing the issue as before:

sh: can't open 'gradlew': No such file or directory

I have tried using the jdk21-alpine image instead and setting the command to ls to see if it would list the files, the build succeeds but ls shows nothing, so still seems like it has nothing in the directory its working in

Edit: I tried to look on the GitHub readme for the docker plugin and found the mount-checkout, but that is default true: GitHub - buildkite-plugins/docker-buildkite-plugin: 🐳📦 Run any build step in a Docker container

4

No worries @ldev654 :slightly_smiling_face:

Hmm thats quite interesting; the plugin’s default working directory can be changed if its needed, just defaults to /workdir in unspecified scenarios if you’d potentially like it to be changed. What host OS is this being cloned into/run on? For reference. my example did use gradle:jdk17 but it should be similar for upper versions.

On the mounting; are you running an ls in the same container as the .gradlew command - something akin to command: ls && sh ./gradlew clean build? The files cloned in on the host (checkout hook) should be mapped over through the volume mount as above.

5

It’s very strange, even after following docs etc it didn’t work @james.s

The agent is running on Ubuntu 22.04.2 in a wrapped docker container. This is the Dockerfile

FROM buildkite/agent

ADD hooks /buildkite/hooks/

ENV BUILDKITE_AGENT_TOKEN=default_token

CMD ["start"]

Then I also have a hooks folder, which contains environment (used for setting the git ssh key) so no other configuration.

Could it be anything with the agent itself I have done wrong or should do? Thank you

6

Thanks for confirming @ldev654 :+1:

Assuming that the above wrapped Dockerfile is the agent thats spun up on the host that that is also registered with Buildkite and from which it is running said steps; we’ll have a Docker-in-Docker scenario.

Are you able to run the actual agent bootstrap on the actual host OS that said image above runs? From there; it will run the actual gradle:jdk21 image as a container; rather than the agent container itself that is pulling the Gradle image in/running it in an embedded container.

From the first premise; my tests above were on a Mac running host (would be similar for others like Ubuntu etc) - for which the actual agent was running on and creating said Gradle container/mounting the checked out repository with the Gradle artefacts :slightly_smiling_face:

7

Thank you @james.s!

So instead of running the docker image of the agent, I am running it as per the docs for Linux directly on the OS and it now works - so it seems like the issue was with what I had configured for the agent.

Thanks again for helping :slight_smile:

8

Always welcome @ldev654!

Its more the case as I thought above; where if you were running the agent on the host itself as a container; it technically can accept work etc as per normal, its more just the case where other images are ran inside that; DinD - and volume mounting between these layers - which can be a bit tricky to navigate!

Glad to have helped, let us know if anything else arises :+1:

1 Like
9

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.