CentOS partition reporting out of space

Question

I have CentOS server with 7 partitions and 2 of them are LVM partitions one is divided as 2 LVM partitions. While working with the production software the disk suddenly gets full in a folder that is stored in one of the LVM partitions that are together in one of the 7 normal partitions. My / partition is only 40G while the /somedir LVM partition is 250G of space, why is the system reporting disk full when the files in the LVM partition grow near the size of the / partition??

the df -h command throws:

/dev/sda2              39G  6,9G   30G  19% /
tmpfs                 3,9G  1,4M  3,9G   1% /dev/shm
/dev/mapper/vgdata-bcks
                      247G  172G   63G  74% /backups
/dev/sda1             485M  310M  151M  68% /boot
/dev/mapper/vgusers-home
                      386G  3,4G  363G   1% /home
/dev/sda5              14G   36M   13G   1% /tmp
/dev/mapper/vgdata-usrs
                      247G   30G  205G  13% /usr1
192.168.11.70:/data   2,0T  249G  1,8T  13% /mnt/SATANAS
//192.168.11.240/Spooler
                      233G   57G  177G  25% /mnt/Fiscaldir
//192.168.11.223/Catalogos
                       96G   44G   53G  46% /media/Catalogos
//192.168.11.204/compartidos
                       98G   42G   51G  46% /mnt/Reportes
/dev/sdb1             3,7T   31G  3,7T   1% /media/Seagate Backup Plus Drive

sorry it was not formatted before

Can you show us the output of `df -h` and `cat /etc/fstab` ? — Zypher, Oct 10 '13 at 15:53
I would try running `strace > ~/trace.log` and see what the software is actually doing. Seems like it's doing operations that puts data on the '/' partition — Zypher, Oct 10 '13 at 16:02
sorry, I would like to do that but I don't really know which app is causing this, I know which app crashes but it is a database that is working properly, except when this happens — Rafael Asprilla, Oct 10 '13 at 16:05
Could it be an issue with the sizes of the partitions and the way the OS see the LVM and the regular partitions on the server? — Rafael Asprilla, Oct 10 '13 at 16:08

score 3 · Answer 1 · edited Oct 10 '13 at 16:27

3

I believe you've run out of inodes on the partition.
If you do df -i you will probably find the inode usage is 100%.

edited Oct 10 '13 at 16:27

voretaq7

79,879
17
130
214

answered Oct 10 '13 at 16:18

user105566

29
1
5

Ok it could be, how can I increase the amount of inodes on the main partition... – Rafael Asprilla Oct 10 '13 at 17:59
delete files from partition/volume group! – user105566 Oct 10 '13 at 18:20

score 1 · Accepted Answer · edited Oct 10 '13 at 20:01

First try to figure out if there is any error in the partition using "dmesg" command. If you find any error, and if you are not much familiar to fix the issue, a simple reboot may fix the issue.
If it is truly a inode related issue, you may reduce your reserved space. It will give some space and also inode as well. (though it is not recommended for production environment. As soon as possible revert it back to 2% or 3%).
```
tune2fs -m 0 /dev/sda1
tune2fs 1.41.12 (17-May-2010)
Setting reserved blocks percentage to 0% (0 blocks)
```
Find zero sized or unused files using find command and remove it after analyzing the files.
If possible take backup of the entire file system and create filesystem again using the command,
```
mkfs.ext4 -i </dev/destination>
```

to specify the number of inodes and then restore the original contents into this.

CentOS partition reporting out of space

2 Answers2