4

I am trying to mount the root of my repository to the docker container in a Azure Pipeline job. On my Windows local I can set the source as $(pwd) successfully, but on my Ubuntu hosted agent I get the following error when using $(Build.SourcesDirectory):

docker: Error response from daemon: invalid mount config for type "bind": bind mount source path does not exist: /var/vsts/28/s.

The path /var/vsts/28/s is correct, so what is the issue?

Yaml definition:

jobs:
- job: Run
  pool:
    name: 'Docker'
  steps:
  - task: Docker@2
    displayName: Login to ACR
    inputs:
      command: login
      containerRegistry: acrServiceConnection
  - task: PowerShell@2
    inputs:
      displayName: Run pulumi go
      targetType: inline
      script: |
        docker run --rm --mount type=bind,source=$(Build.SourcesDirectory),target=/project myimage:latest

EDIT:

When using an MS hosted agent the above works just fine. This leads me to believe it's a permission issue at the checkout location. Any suggestions for this?

I get the same result on our self-hosted and MS agents when running ls -l:

drwxr-xr-x 8 root root 4096 Sep  9 16:09 folder
-rw-r--r-- 1 root root 1347 Sep  9 16:09 file
-rw-r--r-- 1 root root  178 Sep  9 16:09 ..
-rw-r--r-- 1 root root 5457 Sep  9 16:09 ..
danielorn
  • 5,254
  • 1
  • 18
  • 31
Dave New
  • 38,496
  • 59
  • 215
  • 394
  • Maybe not a real answer, but can't you just git clone the project into the container? Since that's essentially the stuff that will be in the directory you are trying to mount. – The Fool Sep 09 '21 at 17:03
  • 1
    1. check if the same pipeline works on a shared agent - I tried it and it works just fine. 2. search checkout task logs for the actual directory where code is checked out (will say something like: `Initialized empty Git repository in /home/vsts/work/1/s/`. 3. Maybe it's an issue with FS permissions? Try adding a step with a script that would list all files in `$(Build.SourcesDirectory)` and maybe modify/remove one file to verify? – qbik Sep 10 '21 at 04:41
  • @TheFool that won't be possible for our use case unfortunately. – Dave New Sep 10 '21 at 07:25
  • @qbik There is no indication of where the repo is initialised too. See post for details. – Dave New Sep 10 '21 at 07:27
  • Similar issue: https://stackoverflow.com/questions/68172042/docker-volume-mounts-not-working-in-azure-devops-pipeline – Dave New Sep 10 '21 at 11:51
  • @qbik It must be a permission issue (or something) since this works fine on a MS hosted agent. Any suggestion on how to fix this perhaps? – Dave New Sep 10 '21 at 11:53
  • 1
    [this question](https://stackoverflow.com/questions/61316142/invalid-mount-config-for-type-bind-bind-mount-source-path-does-not-exist-ho) mentions changing `--mount` to `-v` (which tries to create the path if it doesn't exist). That should at least give you some clue - maybe it will fail with a different error or it'll succeed and create an empty directory (then the path is wrong). – qbik Sep 11 '21 at 04:29
  • @qbik the folder is created but it is empty. So somehow neither `--mount` nor `-v` can successfully mount. – Dave New Sep 13 '21 at 07:49
  • If you use some random self created directory with `chmod -R 777` and some files in it, does it work? – Noam Yizraeli Sep 14 '21 at 10:32
  • @DaveNew how is your self hosted agent configured? Is the agent running as a container itself or as a normal process on ubuntu? – danielorn Sep 18 '21 at 18:29

2 Answers2

4

It is not specifically mentioned how the Self Hosted agent is setup, and it is possible to run an Ubuntu agent both installed directly into a VM, but also as a Docker container.

If the agent is running as a Docker container the error is likely coming from that when the inner container starts it references a path that exists only in the outer container, but not on the host.

When you run a Docker container within a pipeline and that pipelines executes within an agent also started a a docker container the following happens

|=============================================|
|                    HOST                     |
|   |------------------------------------|    |
|   |       Outer container (Agent)      |    |  
|   |                                    |    |
|   |------------------------------------|    |
|                                             |
|   |------------------------------------|    |
|   |       Inner container (Agent)      |    |  
|   |       (Started from pipeline)      |    |
|   |------------------------------------|    |
|                                             |
|=============================================|

When a Docker container runs inside another Docker container, they both use host's docker deamon and thus all mount paths reference the host, regardless of if a new container is launched from the host or the outer container

Example 1: Mount a path from the host into the outer container

docker run ... -v <path-on-host>:<path-on-outer-container> ...

Example 2: Mount a path from the host into the inner container

docker run ... -v <path-on-host>:<path-on-inner-container> ...

Example 2: Mount a path from the outer container into the inner container Mounting paths from outer container into the inner one is not possible without a workaround, since both containers run on the daemon on the host.

One could either make sure that there is a "shared" space on the Host that mounts into both the outer container and the inner container. (Be careful to always specify a path valid on the host even if the inner container is started from within the outer)

Another option is the one described in the section Mounting volumes using Docker within a Docker container in the Microsoft documentation on Docker agents:

Declare an ENV variable when the outer container is started:

docker run ... --env DIND_USER_HOME=$HOME ...

After this, we can start the inner container from the outer one:

docker run ... -v $DIND_USER_HOME:<path-on-inner-container> ...
danielorn
  • 5,254
  • 1
  • 18
  • 31
  • can you tell me how does this work? What is actually `$HOME` when starting the outer container? I want to mount `$(System.DefaultWorkingDirectory)` from outer container (agent) to inner container (my own container) but that doesn't seem to work – faizan Feb 22 '23 at 15:00
1

As stated in the comments, this is probably a permission issue. In your edit you show us that the owner of the files is root. In windows this shouldn't really matter because NTFS doesn't support permissions the same way they work on Linux/ext4, it cannot store the permissions, so on windows they are most likely just ignored.

As suggested, try prefixing your docker run with a chown like this:

sudo chown -R $USER $(Build.SourcesDirectory) && docker run ....

or make sure your code is checked in into VC with the correct permissions set.

jaaq
  • 1,188
  • 11
  • 29
  • `$USER` is empty so I get "chown: missing operand after ‘/var/vsts/31/s’". Note that this does work in the hosted Linux agents. Just not in our self-hosted agents. – Dave New Sep 15 '21 at 12:43
  • by "this" you mean this solution or your use-case that works? Does your build system expose the user-name of the runner maybe? Or maybe it is as simple as changing the double quotes into single quotes to make sure the variables are lazily evaluated? – jaaq Sep 15 '21 at 12:56