1

I was using Docker on my CentOS machine for a while and had lot of images and containers (around 4GBs). My machine has 8GBs os storage and I kept getting an error from devicemapper whenever trying to remove a Docker container or Docker image with docker rm or docker rmi. The error was: Error response from daemon: Driver devicemapper failed to remove root filesystem. So I stopped the Docker service and tried restarting it, but that failed due to devicemapper. After that I uninstalled Docker and removed all images, containers, and volumes by running the following command: rm -rf /var/lib/docker. However, after running that it does not seem like any space was freed up:

$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1      8.0G  7.7G  346M  96% /
devtmpfs        1.8G     0  1.8G   0% /dev
tmpfs           1.8G     0  1.8G   0% /dev/shm
tmpfs           1.8G  193M  1.6G  11% /run
tmpfs           1.8G     0  1.8G   0% /sys/fs/cgroup
tmpfs           361M     0  361M   0% /run/user/1000

$ du -ch -d 1 | sort -hr
3.6G    total
3.6G    .
1.7G    ./usr
903M    ./var
433M    ./home
228M    ./opt
193M    ./run
118M    ./boot
17M     ./etc
6.4M    ./tmp
4.0K    ./root
0       ./sys
0       ./srv
0       ./proc
0       ./mnt
0       ./media
0       ./dev

Why does df tell me I am using 7.7G whereas du tells me I am using 3.6G? The figure that du gives (3.6G) should be the correct one since I deleted everything in /var/lib/docker.

pcsram
  • 597
  • 2
  • 5
  • 12
  • Cleaner to remove docker images with `docker rmi`. You may not have properly removed the images or the interim built containers. – Matthew Schuchard Jul 08 '16 at 17:28
  • I could not remove any containers or images with `docker rm` or `docker rmi`. I kept getting the error `Error response from daemon: Driver devicemapper failed to remove root filesystem`. Afterwards, I just uninstalled Docker altogether and deleted all images, containers, and volumes by running the following command: `rm -rf /var/lib/docker`. – pcsram Jul 08 '16 at 18:22
  • Humm. Despite docker service being stopped some running processes (or something in kernel?) still using files under /var/lib/docker. `lsof |grep deleted` might have shown files in use? Killing off the processes would allow the files to be deleted. – gaoithe Jan 22 '20 at 12:21
  • Also use -x arg to du to get it to stay on one filesystem. `du -h -d 0 -x / |sort -h` as there may be more than one filesystem mounted in /. – gaoithe Jan 22 '20 at 12:22

1 Answers1

0

I had a similar issue. This ticket was helpful.

Depending on the file system you are using, you will want to use either fstrim, zerofree or add the drive to another machine or and use use xfs_repair

If your file system is xfs and you used xfs_repair then after running that command there should be a lost+found directory at the root of the drive that contains all the data that was taking upspace but unreachable.

You can then delete that and it will actually be reflected in du.

Daniel Rucci
  • 2,822
  • 2
  • 32
  • 42