141

I was using Docker in the old way, with a volume container:

docker run -d --name jenkins-data jenkins:tag echo "data-only container for Jenkins"

But now I changed to the new way by creating a named volume:

 docker volume create --name my-jenkins-volume 

I bound this new volume to a new Jenkins container. The only thing I've left is a folder in which I have the /var/jenkins_home of my previous jenkins container. (by using docker cp) Now I want to fill my new named volume with the content of that folder.

Can I just copy the content of that folder to /var/lib/jenkins/volume/my-jenkins-volume/_data?

Makoto
  • 104,088
  • 27
  • 192
  • 230
DenCowboy
  • 13,884
  • 38
  • 114
  • 210

7 Answers7

200

You can certainly copy data directly into /var/lib/docker/volumes/my-jenkins-volume/_data, but by doing this you are:

  • Relying on physical access to the docker host. This technique won't work if you're interacting with a remote docker api.

  • Relying on a particular aspect of the volume implementation would could change in the future, breaking any processes you have that rely on it.

I think you are better off relying on things you can accomplish using the docker api, via the command line client. The easiest solution is probably just to use a helper container, something like:

docker run -v my-jenkins-volume:/data --name helper busybox true
docker cp . helper:/data
docker rm helper
jcollado
  • 39,419
  • 8
  • 102
  • 133
larsks
  • 277,717
  • 41
  • 399
  • 399
  • 3
    Regarding your second bullet, you can run `docker volume inspect my-jenkins-volume --format '{{.Mountpoint}}'` to get it's physical location programmatically. Still doesn't feel like a great idea though. – c24w Oct 18 '17 at 14:35
  • 16
    This helper container never needs to actually run. It would be sufficient to simply create it, then run `docker cp` and then remove it. – Alex Oct 27 '17 at 14:36
  • You can't exec into that container to see the results or modify the files manually. – CodeOrElse Aug 01 '18 at 08:40
  • 4
    Note that listing `/var/lib/docker/volumes/my-jenkins-volume/_data` when using [Docker for Mac](https://docs.docker.com/docker-for-mac/) doesn't work because files are stored inside the [xhyve virtual machine](https://github.com/mist64/xhyve). See https://forums.docker.com/t/var-lib-docker-does-not-exist-on-host/18314 – Ortomala Lokni Aug 22 '18 at 12:35
  • What does 'true' do in the run statement above? – Zuabi Sep 05 '18 at 21:08
  • 2
    True is explained here https://stackoverflow.com/questions/29762231/how-can-i-understand-the-role-of-bin-true-command-in-docker-run-command – Zuabi Sep 05 '18 at 21:11
  • @Alex How would that work? `docker create` instead of `docker run`? – David Moles Mar 27 '19 at 20:11
  • @DavidMoles, yes, `docker create` instead of `docker run` – Alex Mar 29 '19 at 19:28
59

You don't need to start some container to add data to already existing named volume, just create a container and copy data there:

docker container create --name temp -v my-jenkins-volume:/data busybox
docker cp . temp:/data
docker rm temp
Dmytro Melnychuk
  • 2,285
  • 21
  • 23
  • 8
    Provided that busybox's contents are not really needed; you can do this with `hello-world` and it also works. `busybox` is 1.22MB. Instead `hello-world` is 13.3kB. The question is: The same way we can do a Dockerfile FROM scratch, could we do a "docker container create" with "nothing" as the image as we only want to just "mount" the volume and never start the container? – Xavi Montero May 03 '20 at 18:30
  • 2
    +1 for this solution over the top voted but the correct syntax for `docker cp` is `docker cp [OPTIONS] CONTAINER:SRC_PATH DEST_PATH` – n1nsa1d00 Oct 13 '20 at 21:54
  • 1
    @MarcoDufal that's one of the syntaxes, but also `docker cp [OPTIONS] SRC_PATH|- CONTAINER:DEST_PATH`. – Sebastiaan May 27 '21 at 06:26
  • 2
    @XaviMontero See [my answer](https://stackoverflow.com/a/68511611/1065654) for a solution that uses a Docker Image of size 0B. It is similar to this answer, but a little bit more complex, since my used image is build locally. This answer is simple and straight to the point. Thanks Dmytro Melnychuk – maiermic Jul 24 '21 at 16:10
  • Good solution!! I like it! – Xavi Montero Jul 24 '21 at 18:29
45

You can reduce the accepted answer to one line using, e.g.

docker run --rm -v `pwd`:/src -v my-jenkins-volume:/data busybox cp -r /src /data
headdab
  • 1,040
  • 9
  • 10
  • 1
    im wondering if the transient nature of /tmp may pose a risk of the container possibly deleting your data before cp completes? http://www.pathname.com/fhs/pub/fhs-2.3.html#TMPTEMPORARYFILES – thurt Jan 17 '18 at 03:19
  • 1
    The link doesn't really clarify the lifetime of files in /tmp. I states: "Programs must not assume that any files or directories in /tmp are preserved between invocations of the program." which implies that the files would survive, but that's a guarantee. The -v option for docker will create a directory in the container if is doesn't exist, so changing /tmp/src to /src will work if you are worried about this potential race condition. I'll edit the answer to reflect this, since there is no downside. – headdab Jan 18 '18 at 16:18
  • 4
    doesn't the `-v \`pwd\`:/src` imply that the command is running on the host? (How can the host map `pwd` if it's a different machine, for example? -- it can't.) If the docker command isn't running on the host this doesn't work. I believe that's why we have docker cp. This seems like it is not "the way" for docker -- it's just a special case that works only when the docker command is running on the host. Do I understand correctly? – Wyck Oct 01 '18 at 21:07
  • Yes, I think you're right. `pwd` must resolve to a file on the host machine. From docker's mount documentation: "In the case of bind mounts, the first field is the path to the file or directory on the host machine." – headdab Oct 02 '18 at 22:32
  • 1
    Therefore, this does not work to copy *your* local files into the container *if* in a remote host, as you are mounting `pwd` which does not even need to exist in the remote host. Instead the solution by Dmytro Melnychuk (create+cp+rm) does copy the local ones into the container no matter where it is running. – Xavi Montero May 03 '20 at 17:50
8

Here are steps for copying contents of ~/data to docker volume named my-vol

Step 1. Attach the volume to a "temporary" container. For that run in terminal this command :

docker run --rm -it --name alpine --mount type=volume,source=my-vol,target=/data alpine

Step 2. Copy contents of ~/data into my-vol . For that run this commands in new terminal window :

cd ~/data docker cp . alpine:/data

This will copy contents of ~/data into my-vol volume. After copy exit the temporary container.

Namig Hajiyev
  • 1,117
  • 15
  • 16
8

You can add this BASH function to your .bashrc to copy files to a existing Docker volume without running a container

# Usage: copy-to-docker-volume SRC_PATH DEST_VOLUME_NAME [DEST_PATH]
copy-to-docker-volume() {
  SRC_PATH=$1
  DEST_VOLUME_NAME=$2
  DEST_PATH="${3:-}"
  # create smallest Docker image possible
  echo -e 'FROM scratch\nLABEL empty=""' | docker build -t empty -
  # create temporary container to be able to mount volume
  CONTAINER_ID=$(docker container create -v "${DEST_VOLUME_NAME}":/data empty cmd)
  # copy files to volume
  docker cp "${SRC_PATH}" "${CONTAINER_ID}":"/data/${DEST_PATH}"
  # remove temporary container
  docker rm "${CONTAINER_ID}"
}

Example

# create volume as destination
docker volume create my-volume
# create directory to copy
mkdir my-dir
echo "hello file1" > my-dir/my-file-1
# copy directory to volume
copy-to-docker-volume my-dir my-volume
# list directory on volume
docker run --rm -it -v my-volume:/data busybox ls -la /data/my-dir
# show file content on volume
docker run --rm -it -v my-volume:/data busybox cat /data/my-dir/my-file-1

# create another file to copy
echo "hello file2" > my-file-2
# copy file to directory on volume
copy-to-docker-volume my-file-2 my-volume my-dir
# list (updated) directory on volume
docker run --rm -it -v my-volume:/data busybox ls -la /data/my-dir
# check volume content
docker run --rm -it -v my-volume:/data busybox cat /data/my-dir/my-file-2
maiermic
  • 4,764
  • 6
  • 38
  • 77
  • This is great. Always felt weird to me that there wasn't a way to do this without busybox or alpine. Granted it is more commands, but doesn't download/spinup unnecessary resources – Schmorrison Sep 19 '21 at 15:00
  • Great solution! But i think there is an error in the script function: my-volume should be $DEST_VOLUME_NAME – vanem Aug 09 '23 at 07:27
  • @vanem Thanks, good catch! I fixed the code. – maiermic Aug 15 '23 at 18:07
5

If you don't want to create a docker and you can access as privileged user to , simply do (on Linux systems):

docker volume create my_named_volume
sudo cp -p . /var/lib/docker/volumes/my_named_volume/_data/

Furthermore, it also allows you to access data in docker runtime or also with docker containers stopped.

Alejandro Galera
  • 3,445
  • 3
  • 24
  • 42
5

If you don't want to create a temp helper container on windows docker desktop (backed by wsl2) then copy the files to below location

\\wsl$\docker-desktop-data\version-pack-data\community\docker\volumes\my-volume\_data

here my-volume is the name of your named volume. browse the above path from address bar in your file explorer. This is a internal network created by wsl in windows.

enter image description here

Note: it might be better to use docker API like mentioned by larsks, but I have not faced any issues on windows.

Similarly on linux files can be copied to

/var/lib/docker/volumes/my-volume/_data/
ns15
  • 5,604
  • 47
  • 51