No space left on device

Question

I am getting the error "No space left on device" when i tried to scp some files to a centos machine,

tried to check:

[root@...]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol01 18G   18G     0 100% /

And when I do

du -sh / -> it gives only 5G

[... ~]$ df -i /
Filesystem            Inodes   IUsed   IFree IUse% Mounted on
/dev/mapper/VolGroup00-LogVol01
                     4685824  209516 4476308    5% /

seems like file system is full. How can i find which one is taking these much size?

Maybe you get more/better answers on 'Superuser' or 'Unix & Linux'. Please show your scp command and the complete df list. Are you sure that you copy to /dev/mapper/VolGroup00-LogVol01? — smartmeta, Jul 10 '14 at 08:28
i tried to start some services in that server. it is also giving the same error — Futuregeek, Jul 10 '14 at 08:32
Silly me, my target that was full was a symlink to another drive that was full! — Sridhar Sarnobat, Mar 22 '22 at 22:51

score 78 · Accepted Answer · answered Jul 10 '14 at 09:01

78

Such difference between the output of du -sh and df -h may happen if some large file has been deleted, but is still opened by some process. Check with the command lsof | grep deleted to see which processes have opened descriptors to deleted files. You can restart the process and the space will be freed.

answered Jul 10 '14 at 09:01

VolenD

3,592
18
23

3

I tried to kill those processes and it is working . Thanks! – Futuregeek Jul 10 '14 at 09:10
3

How do you restart the process? @user3584460 – IsraGab Mar 20 '16 at 18:56
@IsraGab If it is a daemon, just stop and start it. If it is not a daemon, just kill with with `kill` or `killall` and then start it. – VolenD Mar 21 '16 at 10:36
How would I use killall to processes that are associated with deleted files? – Dr.PP Jul 14 '18 at 13:47
@PatricioPiedra First you have to identify the names of the processes (using `lsof` as mentioned in my answer) and then use `killall ` – VolenD Jul 25 '18 at 09:01
I read about `lsof / | grep deleted` on a page but could not understand what it does. Thank you for your answer here. – Umesh Kaushik Jan 03 '19 at 05:52
This just causes process to restart. Disk is still in 100% use.. – Matej J Oct 25 '19 at 08:28
Nothing happen , no description given after I execute this command – Mark Anthony Libres Nov 03 '21 at 00:38

score 65 · Answer 2 · edited Jan 05 '17 at 13:04

65

Maybe you are out of inodes. Try df -i

                     2591792  136322 2455470    6% /home
/dev/sdb1            1887488 1887488       0  100% /data

Disk used 6% but inode table full.

edited Jan 05 '17 at 13:04

tripleee

175,061
34
275
318

answered Nov 17 '16 at 03:17

BenJLI

650
5
4

1

can we add some more inodes ? :)) (partly joking, partly serious) – Ciprian Tomoiagă Feb 16 '18 at 08:17
https://unix.stackexchange.com/questions/26598/how-can-i-increase-the-number-of-inodes-in-an-ext4-filesystem tl;dr no, not really - you can only add more inodes by creating a new filesystem with more inodes and then copying the existing filesystem onto the new filesystem. – Chris Browne Jan 04 '21 at 12:24
1

You used 6% of the /home but the /dev/sbd1 used 100% - that is the problem. Most likely you have way too many log files. Find large files by running: `find / \( -path /proc -prune -a -path /dev -prune \) -o -type f -size +100M -exec ls -s1 {} \; 2>/dev/null| sort -n -r | head -n 20` – Robert Guice May 10 '21 at 17:39

score 15 · Answer 3 · answered Aug 25 '15 at 10:33

15

To list processes holding deleted files a linux system which has no lsof, here's my trick:

    pushd /proc ; for i in [1-9]* ; do ls -l $i/fd | grep "(deleted)" && (echo -n "used by: " ; ps -p $i | grep -v PID ; echo ) ; done ; popd

answered Aug 25 '15 at 10:33

Bence Frenyó

301
3
5

score 1 · Answer 4 · answered Apr 12 '17 at 07:34

1

You can execute the following commands

lsof / |grep deleted

kill the process id's, which free up the disk space.

answered Apr 12 '17 at 07:34

user7855012

37
1

Not working..... – Matej J Oct 25 '19 at 08:29
1

Use `lsof / | grep DEL` on Ubuntu Linux. – peterhil Sep 30 '21 at 20:00

No space left on device

4 Answers4

Linked

Related