Linux no space left on device issue

Question

We have SUSE Linux OS:

$ uname -a
Linux 3.0.13-0.27-default #1 SMP Wed Feb 15 13:33:49 UTC 2012  x86_64 x86_64 x86_64 GNU/Linux

Only for a specific file the file creation fails with error no space left on device. We are able to create other files, what has gone wrong? In the current scenario:

Inode usage is only 47%

$ df -i .
Filesystem               Inodes    IUsed    IFree IUse% Mounted on
/dev/mapper/appvg-dsfs 39321600 18218000 21103600   47% /dsfs

Disk usage is only 82%

$ df -h .
Filesystem              Size  Used Avail Use% Mounted on
/dev/mapper/appvg-dsfs  591G  457G  105G  82% /dsfs

File creation of one specific file fails in this case JDE_GL_INCR_STG_201504_181.log.load.058

$ echo "a" > JDE_GL_INCR_STG_201504_181.log.load.058
-bash: JDE_GL_INCR_STG_201504_181.log.load.058: No space left on device
echo "a" > testfile

$ mv JDE_GL_INCR_STG_201504_181.log.load.058 tempfile
mv: cannot stat `JDE_GL_INCR_STG_201504_181.log.load.058': No such file or directory

File creation of other files is possible

Not sure if this matters, but what filesystem are you using? For example XFS has some issues with full partitions, although your disk is not that full. — Cristian Ciupitu, May 06 '15 at 13:27
What file system is it? (Maybe it is a hardware error? What does dmesg say?) — Konrad Gajewski, May 06 '15 at 13:29
Can you `echo "a">testfile; mv testfile JDE_GL_INCR_STG_201504_181.log.load.058` (ie, the reverse of the mv above)? — MadHatter, May 06 '15 at 13:45
Could this be a hashing issue as discussed at http://serverfault.com/questions/539536/no-space-left-on-device-error-when-writing-certain-session-filenames ? — Paul Haldane, May 06 '15 at 13:54
Quotas or AppArmor (similar to SELinux) can also cause this. — Paul Gear, May 13 '15 at 07:37
seems an hash-collision issue as described [here][1] http://serverfault.com/questions/539536/no-space-left-on-device-error-when-writing-certain-session-filenames [1]: http://serverfault.com/questions/539536/no-space-left-on-device-error-when-writing-certain-session-filenames — user3546624, Aug 21 '15 at 21:11
Seems an hash-collision issue as described here: http://serverfault.com/questions/539536/no-space-left-on-device-error-when-writing-certain-session-filenames — user3546624, Aug 21 '15 at 21:14
Do you have a vServer or managed Server? The go an check your quotas for this account. You may exceed the disk space granted for the user trying to execute the ```cp```or ```mv``` commands. — WeSee, Feb 27 '16 at 13:34

score 1 · Answer 1 · answered Jul 23 '16 at 06:09

Maybe you run out of Semaphores. Try this as root

ipcs -s

and check if there are a lot of semaphores owned by the same user. If so run the following command

ipcs -s |grep username |awk '{print $2}'

in the above command replace the username withn the actual user.

score 0 · Answer 2 · answered Feb 13 '16 at 10:02

0

really do look like hash-collision issue. Have u tried creating this exact same file in any other empty directory on the same partition to test if it is really hash-collision issue or something else !!

answered Feb 13 '16 at 10:02

Vinay Kumar Setty

11
4

score 0 · Answer 3 · edited Jul 17 '16 at 07:44

I have seen issues of this nature when I have mounted a faulty or corrupted drive. If you have smartctl on the host, run it against the device (smartctl -a /dev/<device>). Different drives have different data, but look for things like: uncorrected errors, multi_zone_error_rate, current_pending_sector, raw_read_error_rate, or if it says the drive has failed. Those are all indicators that the is not in the best state.

score -3 · Answer 4 · answered May 06 '15 at 16:15

-3

Firstly check the size of the file you are trying to create. The file might be very big. Ok I know the chances of the above being the issue may be low, rename the file to a different name or create it with different file name..

answered May 06 '15 at 16:15

Kevin Kaburu

97
2

1

Please read again my question. The file creation command is mentioned there. `echo "a" > JDE_GL_INCR_STG_201504_181.log.load.058` By this method the file size is not over a kb. Even touch gives same error `touch JDE_GL_INCR_STG_201504_181.log.load.058` – Nishant Bhardwaj May 07 '15 at 06:23

Linux no space left on device issue

4 Answers4