Docker volume mounts not working in Azure DevOps Pipeline

Question

Docker volume mounts not working in Azure DevOps Pipeline, please find my code below:

I tried two approaches to run my docker container in the pipeline - please refer below - both returning empty volume - volume mount not happening. I'm not sure what mistake I'm doing here. It would be really appreciated if someone can help me to fix this issue.

I would like to mount run.sh, test.sh and test.txt under /test

In entrypoint.sh - I'm listing all the files inside docker - but it's returning empty - not mounted all these files run.sh, test.sh and test.txt

I'm struggling for the last two days to fix this issue but not getting any resolution - any help would be really appreciated.

This is my folder structure:

my-app/
├─ test/
│  ├─ run.sh
│  ├─ test.sh
│  ├─ test.txt
├─ azure-pipeline.yml

test.sh

#!/bin/bash

rootPath=$1
echo "Root path: $rootPath"
./run.sh $rootPath

run.sh

#!/bin/bash

echo "starting run script"

NAME="testApp"
IMAGE="sample/test-app:1.1"
ROOTPATH=$1

echo "$ROOTPATH"
# Finally run
docker stop $NAME > /dev/null 2>&1
docker rm $NAME > /dev/null 2>&1
docker run --name $NAME -i -v $ROOTPATH:/test -w /test $IMAGE

azure-pipeline.yml (Approach -1)

trigger:
  - none

jobs:
  - job: test
    pool:
      name: my-Linux-agents
    displayName: Run tests
    steps:
      - task: Bash@3
        displayName: Docker Prune
        inputs:
          targetType: inline
          script: |
            docker system prune -f -a

      - task: Docker@2
        displayName: Docker Login
        inputs:
          containerRegistry: myRegistry w/ asdf
          command: login

      - task: Bash@3
        displayName: Execute Sample Java
        inputs:
          targetType: filePath
          filePath: 'test/test.sh'
          arguments: '$PWD'
          workingDirectory: test

azure-pipeline.yml (Approach -2)

trigger:
  - none

jobs:
  - job: test
    pool:
      name: my-Linux-agents
    displayName: Run tests
    steps:
      - task: Bash@3
        displayName: Docker Prune
        inputs:
          targetType: inline
          script: |
            docker system prune -f -a

      - task: Docker@2
        displayName: Docker Login
        inputs:
          containerRegistry: myRegistry w/ asdf
          command: login

      - bash: |
          echo "Executing docker run command"
          echo $(Build.SourcesDirectory)
          echo $PWD
          docker run --name testApp -i -v $(Build.SourcesDirectory):/test -w /test sample/test-app:1.1

My Docker Image - files Dockerfile

FROM alpine:3.12

COPY entrypoint.sh /
RUN echo "hello"

ENTRYPOINT ["/entrypoint.sh"]

entrypoint.sh

#!/bin/sh

echo "START Running Docker"
echo "Listing Files"
ls -la

I tried your code and in Approach 2 I can get the files inside my container. can you share the logs? — Shayki Abramczyk, Jun 29 '21 at 10:58
It looks like a permission issue... Sure there's nothing at all in docker logs? — Diego Rafael Souza, Jul 05 '21 at 02:19
@MikeMarsh I am experiencing the exact same issue - did you figure it out in the end? — Dave New, Sep 10 '21 at 10:53
@MikeMarsh I am unaable to reproduce tour scenarios, both are working fine. At least on the Microsoft Hosted `ubuntu-latest` pool. Are you running this on a self hosted agent? If so I the problem is likely with the configuration on the agent itself. You could try switching to a Microsoft Hosted agent to verify that your pipeline config and scripts are correct. — danielorn, Sep 18 '21 at 16:41

danielorn · Answer 1 · 2021-09-20T05:56:12.833

TL;DR

Docker volume mounts work in Azure DevOps Pipelines (at least on the Microsoft Hosted Agents).

As can be seen below both the approaches described in the OP works with the ubuntu-latest agent pool. If self hosted agents are used in my-Linux-pool the problem is likely with them rather than with the Dockerfile or pipeline config that was shared in the original post.

Please see the full working demo with git repo and Pipeline

EDIT: If the self hosted pipeline agents are run as docker containers, the problems could come from that the inner container references a path that exists only in the outer container, but not on the host. For more details on how to mount volumes when launching a docker container from another docker container, please see the section on Mounting volumes using Docker within a Docker container from the Azure Pipeline docs, or this answer from Mounting docker run in Azure Pipeline job

Test setup

I have modified the azure-pipelines.yaml to create a fully self contained example:

The following changes were made to the azure-pipelines.yml in the OP

Setting the agentpool to ubuntu-latest rather than my-Linux-agents
Changing the Docker login step to a docker build step that actually builds the image as part of the pipeline. (Not necessary, but it makes the pipeline independent of a custom image registry)
Add a step that lists all the files in the repo recursively and their permission (So that we can easily verify that all files in the /test folder are readable by all)
Adding the steps that runs the containers from the two approaches into the same pipeline so that both approaches are demonstrated in the same pipeline

The azure-pipelines.yml now looks like this:

trigger:
  - none

jobs:
  - job: test
    pool:
      vmImage: 'ubuntu-latest'
    displayName: Run tests
    steps:
      - task: Docker@2
        displayName: Build Docker Image
        inputs:
          repository: sample/test-app
          command: build
          Dockerfile: image/Dockerfile
          tags: '1.1'

      - bash: |
          echo $(Build.SourcesDirectory)
          ls -lrtR
        displayName: List files

      - bash: |
          echo "Executing docker run command"
          docker run --name testApp -i -v $(Build.SourcesDirectory)/test:/test -w /test sample/test-app:1.1
        displayName: Run docker inline in pipeline

      - task: Bash@3
        displayName: Run test.sh
        inputs:
          targetType: filePath
          filePath: 'test/test.sh'
          arguments: '$PWD'
          workingDirectory: test

The rest of the files looks the same, the full test setup can be found here

Test Results

When running the pipeline the following output is obtained (Full pipeline run can be found here here

List Files

/home/vsts/work/1/s
.:
total 16
drwxr-xr-x 2 vsts docker 4096 Sep 18 16:51 test
drwxr-xr-x 2 vsts docker 4096 Sep 18 16:51 image
-rw-r--r-- 1 vsts docker  849 Sep 18 16:51 azure-pipelines.yml
-rw-r--r-- 1 vsts docker  198 Sep 18 16:51 README.md

./test:
total 8
-rw-r--r-- 1 vsts docker   0 Sep 18 16:51 test.txt
-rwxr-xr-x 1 vsts docker  72 Sep 18 16:51 test.sh
-rwxr-xr-x 1 vsts docker 258 Sep 18 16:51 run.sh

./image:
total 8
-rwxr-xr-x 1 vsts docker 67 Sep 18 16:51 entrypoint.sh
-rwxr-xr-x 1 vsts docker 87 Sep 18 16:51 Dockerfile

Run docker inline in pipeline

Executing docker run command
START Running Docker
Listing Files
total 16
drwxr-xr-x    2 1001     121           4096 Sep 18 16:51 .
drwxr-xr-x    1 root     root          4096 Sep 18 16:51 ..
-rwxr-xr-x    1 1001     121            258 Sep 18 16:51 run.sh
-rwxr-xr-x    1 1001     121             72 Sep 18 16:51 test.sh
-rw-r--r--    1 1001     121              0 Sep 18 16:51 test.txt

Run test.sh

Root path: /home/vsts/work/1/s/test
starting run script
/home/vsts/work/1/s/test
START Running Docker
Listing Files
total 16
drwxr-xr-x    2 1001     121           4096 Sep 18 16:51 .
drwxr-xr-x    1 root     root          4096 Sep 18 16:51 ..
-rwxr-xr-x    1 1001     121            258 Sep 18 16:51 run.sh
-rwxr-xr-x    1 1001     121             72 Sep 18 16:51 test.sh
-rw-r--r--    1 1001     121              0 Sep 18 16:51 test.txt

score 0 · Answer 2 · answered Sep 24 '21 at 15:08

0

i believe solution is relatively simple, idea derived from here, give this a try:

in pipelines

pool:
  vmImage: 'ubuntu-latest'

in Dockerfile

VOLUME ["/test"]
RUN ls -la >> my.log

and then check log

answered Sep 24 '21 at 15:08

nikhil swami

2,360
5
15

Docker volume mounts not working in Azure DevOps Pipeline

2 Answers2

TL;DR

Test setup

Test Results

Linked