# How to Make Docker Compose Use Same Running Instance?

**URL:** <https://forum.buildkite.community/t/how-to-make-docker-compose-use-same-running-instance/3093>\
**Category:** Docker\
**Created:** [July 22, 2023, 12:46pm UTC](https://forum.buildkite.community/t/how-to-make-docker-compose-use-same-running-instance/3093 "2023-07-22T12:46:25Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![muratgozel](https://avatars.discourse-cdn.com/v4/letter/m/e480ec/32.png) [@muratgozel](https://forum.buildkite.community/u/muratgozel)\
**Post date:** [July 22, 2023, 12:46pm UTC](https://forum.buildkite.community/t/how-to-make-docker-compose-use-same-running-instance/3093/1 "2023-07-22T12:46:25Z")

</div>

Hi again,

After successfully testing and starting with buildkite pipelines, I created a pipeline based on docker-compose:

```yaml
steps:
    - label: "Create docker image"
      plugins:
          - docker-compose#v4.14.0:
                build: app

    - wait

    - label: "Setup"
      command: npm run setup
      plugins:
          - docker-compose#v4.14.0:
                run: app
                env:
                    - AWS_REGION
                    - AWS_ACCESS_KEY_ID
                    - AWS_SECRET_ACCESS_KEY

    - wait

    - label: "Compile"
      command: npm run compile
      plugins:
          - docker-compose#v4.14.0:
                run: app

    - wait

    - label: "Test"
      command: npm run test
      plugins:
          - docker-compose#v4.14.0:
                run: app

    - wait

    - label: "Start"
      command: npm run prod
      plugins:
          - docker-compose#v4.14.0:
                run: app

```

The **setup** and **compile** jobs create files and directories that will be used by the **start** job. But the start job can not find those files because new image run by the agent on each job. Should we concat the commands such as `npm run setup && npm run compile ...` or is there anything we can do to use the same running instance? I preferred the approach above because it seems like a better practice in designing pipelines.

---

<div class="post-metadata">

**Author:** ![benmc](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.buildkite.community/benmc/32/1159_2.png) [@benmc](https://forum.buildkite.community/u/benmc)\
**Post date:** [July 22, 2023, 12:54pm UTC](https://forum.buildkite.community/t/how-to-make-docker-compose-use-same-running-instance/3093/2 "2023-07-22T12:54:05Z")

</div>

Hey @muratgozel!

We advise against this due to the inefficiency of it (uneven distribution of work amongst agents) and potential unreliability it can cause (an agent/instance being offline).

However, it sounds like what you’re looking for is [node affinity](https://buildkite.com/docs/agent/v3/cli-start#run-a-job-on-the-agent-that-uploaded-it-also-known-as-node-affinity); the ability to target the host that ran the `upload` command for all jobs.

Let me know if that’s going to meet your requirements here. Like I said, we advise against it and would actually recommend running `test` and `compile` in the same step, perhaps then uploading to ECR (or another image storage solution) and then using that image in your `prod` step.

Cheers!

---

<div class="post-metadata">

**Author:** ![muratgozel](https://avatars.discourse-cdn.com/v4/letter/m/e480ec/32.png) [@muratgozel](https://forum.buildkite.community/u/muratgozel)\
**Post date:** [July 22, 2023, 1:33pm UTC](https://forum.buildkite.community/t/how-to-make-docker-compose-use-same-running-instance/3093/3 "2023-07-22T13:33:14Z")

</div>

thank you, this was the answer i need, i guess i will redesign my architecture.

---

<div class="post-metadata">

**Author:** ![muratgozel](https://avatars.discourse-cdn.com/v4/letter/m/e480ec/32.png) [@muratgozel](https://forum.buildkite.community/u/muratgozel)\
**Post date:** [August 10, 2023, 12:27pm UTC](https://forum.buildkite.community/t/how-to-make-docker-compose-use-same-running-instance/3093/4 "2023-08-10T12:27:15Z")

</div>

Hi again, so if there is anyone who see this question and look for guidance, I took the foolowing approach. I build and upload the container in the first step and then deploy it with `docker-compose up`. Dockerfile does the app build and test during the first step. So its like pipeline inside pipeline.

```auto
steps:
    - label: "Create container build"
      command: ./devops/cd/image.sh
      env:
        NODE_ENV: production
        CONTAINER_CONN_STR: ghcr.io/path/to/repo

    - wait

    - label: "Deploy"
      command: ./devops/cd/deploy.sh
      concurrency: 1
      concurrency_group: 'path/to/repo/deploy'
      env:
        NODE_ENV: production
        CONTAINER_CONN_STR: ghcr.io/path/to/repo
        APP_PORT: 3030

```

Dockerfile runs build and test stages:

```auto
FROM node:18-alpine3.17 as build

RUN ...

FROM build as test

RUN ...

FROM test as prod

RUN ...

CMD npm run fetch-env && npm run prod

```

---

<div class="post-metadata">

**Author:** ![paula](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.buildkite.community/paula/32/954_2.png) [@paula](https://forum.buildkite.community/u/paula)\
**Post date:** [August 10, 2023, 3:39pm UTC](https://forum.buildkite.community/t/how-to-make-docker-compose-use-same-running-instance/3093/5 "2023-08-10T15:39:20Z")

</div>

Thanks @muratgozel for sharing your approach!

---

<div class="post-metadata">

**Author:** ![paula](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.buildkite.community/paula/32/954_2.png) [@paula](https://forum.buildkite.community/u/paula)\
**Post date:** [August 29, 2023, 5:38am UTC](https://forum.buildkite.community/t/how-to-make-docker-compose-use-same-running-instance/3093/6 "2023-08-29T05:38:46Z")

</div>

Closed on August 29, 2023, 5:38am UTC


