14

I'm trying to create a relatively simple setup to develop and test npm packages. A problem was in the fact, that after you mounted a code volume to the container it replaces node_modules.

I tried a lot of generally logical stuff, mostly aimed to move node_modules to another location and then reference it within configuration files. It works, but the solution is ugly. Also, it's not good practice to install webpack globally, but my solution requires it.

However, after some time I found this solution, which looks elegant, just what I needed, but it also has one problem. I don't understand completely, how it works.

That my version of how everything operates.

  1. Docker reorders volume mounting based on container paths

  2. Docker mounts sub dir volume at first

  3. Docker mounts parent dir volume but due to an unexplained mechanism, it does not override the sub dir volume...

  4. ???

  5. PROFIT. node_modules dir is in place and webpack runs perfectly.

So, I really want to understand how it actually does all of this black magic. Because without this knowledge I feel like I'm missing something important.
So, guys, how it works? Thanks in advance.

services:
  react-generic-form:
    image: react-generic-form:package
    container_name: react-generic-form-package
    build:
      dockerfile: dev.Dockerfile
      context: ./package
    volumes:
      - "./package:/package"
      - "/package/node_modules"
Nikita Balakin
  • 153
  • 1
  • 6
  • Where are you finding the statement about the order `volumes:` are applied? – David Maze Mar 25 '19 at 11:49
  • It's just my logical assumption. This configuration prevents node_modules dir in the container from being deleted by parent dir(/package) mount. The only way it can do this is to mount subfolder volume which is node_modules before its parent. Therefore I think that before mounting volumes with such configuration docker firstly reorders them by their path and mounts subdirectories before their parent dirs. – Nikita Balakin Mar 25 '19 at 16:34

1 Answers1

13

The Docker daemon, when it creates the container, sorts all of the mount points to avoid shadowing. (On non-Windows, this happens in (*github.com/docker/docker/daemon.Daemon).setupMounts.) So, in your example:

  1. The Docker daemon sees that both /package and /package/node_modules contain data that's stored outside the container filespace.
  2. It sorts these shortest to longest.
  3. It mounts /package, as a bind-mount to the named host directory. (First, because it's a shorter path name.)
  4. It mounts /package/node_modules, shadowing the equivalent directory in the previous mount, probably as a bind-mount to a directory with long hex identifier name somewhere in /var/lib/docker/volumes.

You can experiment more with this with a docker-compose.yml file like

version: '3'
services:
  touch:
    image: busybox
    volumes:
      - ./b:/a/b
      - ./a:/a
    command: touch /a/b/c

Notice that whichever order you put the volumes: in, you will get an empty directory ./a/b (which becomes the mount point inside the container), plus an empty file ./b/c (the result of the touch command).

Also note the statement here that the node_modules directory contains data, that should be persisted across container invocations, and has a lifecycle separately from either the container or its base image. Changing the image and re-running docker-compose up will have no effect on this volume's content.

David Maze
  • 130,717
  • 29
  • 175
  • 215
  • You're right, but the problem is still here. If we mount a folder from the host which does not contain node_modules, it will result in container missing node_modules because it should override container folder contents, but not merge it. Comment in code states the right thing, which is correct when you are mounting something from a host, but here node_modules does not exist on a host. node_modules exists in a container as an image layer. I tried to mount only package folder, and it works as expected, node_modules gets wiped and container crushes. So, how node_modules volume prevents this? – Nikita Balakin Mar 26 '19 at 13:34
  • Is this a feature or a bug in Docker? I just don't know. Also, I think I found the short solution which will work just fine and will be logical, but adding a single line in the compose config is much more elegant. – Nikita Balakin Mar 26 '19 at 13:45
  • 1
    So in basically every other language the usual best practice for Docker is to `COPY` the application code into an image; my personal recommendation here would be to not use volumes at all. – David Maze Mar 26 '19 at 13:46
  • Just putting a bare path in `volumes:` tells Docker to create an anonymous volume, if that’s the source of your confusion. – David Maze Mar 26 '19 at 13:47
  • I need to configure dev env, therefore volumes are the only way to get everything running, without them there is no way to do auto compile with file watching. In prod env everything is pretty simple, code updates only as a layer. I'm just curious, because If I remove anonymous node_modules volume everything breaks because the whole container folder replaced by host folder which doesn't contain node_modules, and then build fails. The question is simple. If parent folder volume mounts first how anonymous subfolder volume can prevent deletion of containers subfolder? – Nikita Balakin Mar 26 '19 at 18:05