0

Today the hard drive of our server was suddenly full. The disk usage always stayed around 50 % in the weeks and months before (old data is regularly expunged from the server).

I deleted 10 GB of files in /tmp, which strangely freed 51 GB. Here is what I did: 

root@***:~# df -h
Dateisystem           Size  Used Avail Use% Eingehängt auf
/dev/sda3             139G  137G     0 100% /
tmpfs                 3,9G     0  3,9G   0% /lib/init/rw
udev                  3,9G  116K  3,9G   1% /dev
tmpfs                 3,9G     0  3,9G   0% /dev/shm
/dev/sda1             985M   25M  910M   3% /boot

root@***:/var# du -hs *
3,3M    backups
438M    cache
9,4G    lib
4,0K    local
12K     lock
76M     log
24K     mail
4,0K    opt
88K     run
184K    spool
10G     tmp
12K     www

root@***:/var/tmp# find -type f -print0 | xargs -0 rm


root@***:/var/tmp# df -h
Dateisystem           Size  Used Avail Use% Eingehängt auf
/dev/sda3             139G   81G   51G  62% /
tmpfs                 3,9G     0  3,9G   0% /lib/init/rw
udev                  3,9G  116K  3,9G   1% /dev
tmpfs                 3,9G     0  3,9G   0% /dev/shm
/dev/sda1             985M   25M  910M   3% /boot

Any explanation as to why deleting 10 GB in /tmp gave me back 51 GB on the disk? Could this point to an SSD failure? Are there any tools for Debian to test SSD health?

I already have checked syslog. The first entry relating to this incidient is a mysql message:

1:22:02 [ERROR] /usr/sbin/mysqld: Disk is full writing...

So I have absolutely no idea what caused this.

John Gardeniers
  • 27,458
  • 12
  • 55
  • 109
Daniel
  • 1

2 Answers2

2

At a guess something had one or more very large files open, but had deleted them so they were no longer visible in the filesystem. This is a common technique for temporary files.

It was probably blocking trying to write to those file(s) and failing because of the lack of space, and then when you made some space it was able to finish what it was doing and close the file(s) which caused the space they were using to be released.

TomH
  • 1,290
  • 7
  • 10
0

This is probably not related to SSD failure. While it's possible that you could be getting some form of filesystem corruption, I think you'd see much more significant symptoms of that.

I posted here about how to check your SSD's drive health.

As for your other question (how deleting 10GB resulted in 50GB of free space), I think TomH's answer is probably right.

Community
  • 1
Daniel Lawson
  • 5,476
  • 22
  • 27