I am a developer in codiva online ide. Every time the user runs a java program, we start a new container. We use devicemapper storage driver because that is the only way we were able to set an upper limit on the container size.
After a few months of usage, we are seeing that the disk usage is increasing, and /var/lib/docker/devicemapper/devicemapper takes around 2.7GB.
I have deleted all the docker containers that are not running, and any dangling volumes, unfortunately, the disk usage for /var/lib/docker/devicemapper/devicemapper still show up as 2.7GB.
What is the best way to reclaim this space? We don't need any persistent storage, we are okay with deleting all the data, because any data that needs to be stored, we mount the host directory directly, and those files are managed outside of docker.
sudo docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
There are no running or even stopped containers. The docker info results here.
$ sudo docker info
Containers: 1
Images: 14
Server Version: 1.9.1
Storage Driver: devicemapper
Pool Name: docker-202:1-143417-pool
Pool Blocksize: 65.54 kB
Base Device Size: 107.4 GB
Backing Filesystem: ext4
Data file: /dev/loop0
Metadata file: /dev/loop1
Data Space Used: 2.591 GB
Data Space Total: 107.4 GB
Data Space Available: 1.469 GB
Metadata Space Used: 2.531 MB
Metadata Space Total: 2.147 GB
Metadata Space Available: 1.469 GB
Udev Sync Supported: true
Deferred Removal Enabled: false
Deferred Deletion Enabled: false
Deferred Deleted Device Count: 0
Data loop file: /var/lib/docker/devicemapper/devicemapper/data
Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
Library Version: 1.02.77 (2012-10-15)
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 3.13.0-76-generic
Operating System: Ubuntu 14.04.3 LTS
CPUs: 1
Total Memory: 992.5 MiB
Name: ip-172-31-26-50
ID: NIQ5:2NDW:SB77:ZNZU:UO6G:EYLB:JIW7:SQLL:QLWY:LNRM:SU6P:ZS7X
WARNING: No swap limit support
There are no dangling or orphaned volumes.
sudo docker volume ls
DRIVER VOLUME NAME
The only image is java:8
sudo docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
java 8 289bdffc37d7 3 weeks ago 669.2 MB
I had already followed the instructions is https://lebkowski.name/docker-volumes/ and still it is using unnecessary extra space.
Edit1:
I have added the results of docker images -a. This shows a large number unnamed images. The size seems suspicious, as the sum of all these images goes upto 4.5GB, but my entire /var/lib/docker is only around 2.7GB.
$ sudo docker images -a
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
java 8 289bdffc37d7 3 weeks ago 669.2 MB
<none> <none> 5d7cc49c4372 3 weeks ago 668.8 MB
<none> <none> 9e756f3b105a 3 weeks ago 310.9 MB
<none> <none> 7615ebf33bb7 3 weeks ago 310.9 MB
<none> <none> ac079291ad51 3 weeks ago 310.9 MB
<none> <none> cd02ed4609aa 3 weeks ago 310.9 MB
<none> <none> 07bbd1c2007f 3 weeks ago 310.9 MB
<none> <none> 6331ed9a346a 3 weeks ago 310.9 MB
<none> <none> b647ad8e0a08 3 weeks ago 310.9 MB
<none> <none> f23012d4423e 3 weeks ago 310.9 MB
<none> <none> 0d3e866c82f3 3 weeks ago 301 MB
<none> <none> c28cbef85c39 3 weeks ago 169.7 MB
<none> <none> 3f0d3d140ce1 3 weeks ago 125.1 MB
<none> <none> 17bd2058e0c6 3 weeks ago 125.1 MB
On trying to one of them I get the error. $ sudo docker rmi 5d7cc49c4372 Error response from daemon: conflict: unable to delete 5d7cc49c4372 (cannot be forced) - image has dependent child images Error: failed to remove images: [5d7cc49c4372]
On reading about it, is seems that the only required image java:8 depends on these child images.
$ sudo docker history 289bdffc37d7
IMAGE CREATED CREATED BY SIZE COMMENT
289bdffc37d7 3 weeks ago /bin/sh -c /var/lib/dpkg/info/ca-certificates 418.2 kB
5d7cc49c4372 3 weeks ago /bin/sh -c set -x && apt-get update && apt- 357.9 MB
7615ebf33bb7 3 weeks ago /bin/sh -c #(nop) ENV CA_CERTIFICATES_JAVA_VE 0 B
9e756f3b105a 3 weeks ago /bin/sh -c #(nop) ENV JAVA_DEBIAN_VERSION=8u9 0 B
ac079291ad51 3 weeks ago /bin/sh -c #(nop) ENV JAVA_VERSION=8u91 0 B
07bbd1c2007f 3 weeks ago /bin/sh -c #(nop) ENV JAVA_HOME=/usr/lib/jvm/ 0 B
cd02ed4609aa 3 weeks ago /bin/sh -c { echo '#!/bin/sh'; echo 'set 87 B
b647ad8e0a08 3 weeks ago /bin/sh -c #(nop) ENV LANG=C.UTF-8 0 B
6331ed9a346a 3 weeks ago /bin/sh -c echo 'deb http://httpredir.debian. 61 B
f23012d4423e 3 weeks ago /bin/sh -c apt-get update && apt-get install 9.93 MB
0d3e866c82f3 3 weeks ago /bin/sh -c apt-get update && apt-get install 131.2 MB
c28cbef85c39 3 weeks ago /bin/sh -c apt-get update && apt-get install 44.67 MB
3f0d3d140ce1 3 weeks ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0 B
17bd2058e0c6 3 weeks ago /bin/sh -c #(nop) ADD file:76679eeb94129df23c 125.1 MB
That means, the untagged images cannot be removed, and I think by design the total size used should still be ~670MB since docker stores only the diffs for dependent images.
Please let me know how to reduce the disk usage of devicemapper.
