du -skh * in / returns vastly different size from df on centos 5.5

Question

I have a vps slice running centos 5.5 I am supposed to have 15 gigs of disk space, but according to df it seems to double my disk space usage.

when I run du -skh * in / as root i get:

[root@yardvps1 /]# du -skh *
0       aquota.group
0       aquota.user
5.2M    bin
4.0K    boot
4.0K    dev
4.9M    etc
2.5G    home
12M     lib
14M     lib64
4.0K    media
4.0K    mnt
299M    opt
0       proc
692K    root
23M     sbin
4.0K    selinux
4.0K    srv
0       sys
48K     tmp
2.0G    usr
121M    var

this is consistent with what I have uploaded to the machine, and adds up to about 5gigs.

BUT when i run df i get:

[root@yardvps1 /]# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/simfs            15728640  11659048   4069592  75% /
none                    262144         4    262140   1% /dev

it is showing me using almost 12 gigs already.

what is causing this discrepancy and is there anything I can do about it, I planned the server out based on 15 gigs but now it is basically only letting me have about 7 gigs of stuff on it.

thanks.

Answer 1

The most common cause of this effect is open files that have been deleted.

The kernel will only free the disk blocks of a deleted file if it is not in use at the time of its deletion. Otherwise that is deferred until the file is closed, or the system is rebooted.

A common Unix-world trick to ensure that no temporary files are left around is the following:

• A process creates and opens a temporary file

• While still holding the open file descriptor, the process unlinks (i.e. deletes) the file

• The process reads and writes to the file normally using the file descriptor

• The process closes the file descriptor when it's done, and the kernel frees the space

• If the process (or the system) terminates unexpectedly, the temporary file is already deleted and no clean-up is necessary.

• As a bonus, deleting the file reduces the chances of naming collisions when creating temporary files and it also provides an additional layer of obscurity over the running processes - for anyone but the root user, that is.

This behaviour ensures that processes don't have to deal with files that are suddenly pulled from under their feet, and also that processes don't have to consult each other in order to delete a file. It is unexpected behaviour for those coming from Windows systems, though, since there you are not normally allowed to delete a file that is in use.

The lsof command, when run as root, will show all open files and it will specifically indicate deleted files that are deleted:

# lsof 2>/dev/null | grep deleted
bootlogd   2024       root    1w      REG                9,3         58     917506 /tmp/init.0W2ARi (deleted)
bootlogd   2024       root    2w      REG                9,3         58     917506 /tmp/init.0W2ARi (deleted)

Stopping and restarting the guilty processes, or just rebooting the server should solve this issue.

Deleted files could also be held open by the kernel if, for example, it's a mounted filesystem image. In this case unmounting the filesystem or rebooting the server should do the trick.

In your case, judging by the size of the "missing" space I'd look for any references to the file that you used to set up the VPS e.g. the Centos DVD image that you deleted after installing.

Answer 2

Another case which I've come across although it doesn't appear to be your issue is if you mount a partition "on top" of existing files.

If you do so you effectively hide existing files that exist in the directory on the mounted-to partition (the mount point) from the mounted partition.

To fix: stop any processes with open files on the mounted partition, unmount partition, find and move/remove any files that now appear in mount point directory.

Answer 3

I had the same trouble with FreeBSD server. The reboot helped.