Python OSError No space left on device

Question

Im writing a lot of files to a folder, no subdirectories, however after 2.8 million files this error shows up:

with open(bottleneck_path, 'w') as save_file:
OSError: [Errno 28] No space left on device: '/home/user/path/redacted'

I've checked the length of the filename which is only 149 characters long. The file size is supposed to be around 18K.

My operating system is Linux.

df -i:

Filesystem                               Inodes   IUsed    IFree IUse% Mounted on
udev                                    4106923     544  4106379    1% /dev
tmpfs                                   4113022     836  4112186    1% /run
/dev/sda2                              28401664 9008557 19393107   32% /
tmpfs                                   4113022      40  4112982    1% /dev/shm
tmpfs                                   4113022       5  4113017    1% /run/lock
tmpfs                                   4113022      16  4113006    1% /sys/fs/cgroup
/dev/sda1                                     0       0        0     - /boot/efi

df -T

Filesystem                             Type      1K-blocks       Used  Available Use% Mounted on
udev                                   devtmpfs   16427692          0   16427692   0% /dev
tmpfs                                  tmpfs       3290420      22136    3268284   1% /run
/dev/sda2                              ext4      447088512  355325584   69029056  84% /
tmpfs                                  tmpfs      16452088      82448   16369640   1% /dev/shm
tmpfs                                  tmpfs          5120          4       5116   1% /run/lock
tmpfs                                  tmpfs      16452088          0   16452088   0% /sys/fs/cgroup
/dev/sda1                              vfat         523248       3684     519564   1% /boot/efi

du -sh .

56G

ls | wc -l

From my understanding ext4 should be able to handle this just fine.

EDIT:

tune2fs -l /dev/sda2

tune2fs 1.42.13 (17-May-2015)
Filesystem volume name:   <none>
Last mounted on:          /
Filesystem UUID:          cd620466-1f88-400b-acf5-457a9c9544cf
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal ext_attr resize_inode dir_index filetype needs_recovery extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
Filesystem flags:         signed_directory_hash 
Default mount options:    user_xattr acl
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              28401664
Block count:              113587456
Reserved block count:     5679372
Free blocks:              82864623
Free inodes:              25755495
First block:              0
Block size:               4096
Fragment size:            4096
Reserved GDT blocks:      996
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         8192
Inode blocks per group:   512
Flex block group size:    16
Filesystem created:       Wed Mar  1 15:14:22 2017
Last mount time:          Mon Mar 27 13:20:00 2017
Last write time:          Mon Mar 27 13:20:00 2017
Mount count:              35
Maximum mount count:      -1
Last checked:             Wed Mar  1 15:14:22 2017
Check interval:           0 (<none>)
Lifetime writes:          1813 GB
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:           256
Required extra isize:     28
Desired extra isize:      28
Journal inode:            8
First orphan inode:       1312056
Default directory hash:   half_md4
Directory Hash Seed:      e186507d-32b5-49c0-8ce1-09bf2a75d816
Journal backup:           inode blocks

Using

touch /home/user/path/redacted/somefile_1

throws an error but the same filename structure with a different name works fine. eg:

touch /home/user/path/redacted/somefile_2

Are you writing all 2.8 million files as children of said directory, or is there nesting as well? It seems that ext4 has a limit of 64k subdirectories per directory, according to [this question](http://serverfault.com/questions/506465/is-there-a-hard-limit-to-the-number-of-files-a-directory-can-have) — Horia Coman, Mar 29 '17 at 06:59
ext4 should well be able to handle this. There are no subdirectories. — Wesley, Mar 29 '17 at 07:00
Sure, it may be able to handle it but it's still a fairly bad idea that's trivially avoidable. — pvg, Mar 29 '17 at 07:06

score 1 · Answer 1 · answered Mar 29 '17 at 07:02

1

There is no default limit on the maximum inodes for ext4, it depends on the size of the device and the options chosen at creation. Check the existing limits using

tune2fs -l /path/to/device

answered Mar 29 '17 at 07:02

Raymond Hettinger

216,523
63
388
485

score 0 · Accepted Answer · answered Mar 29 '17 at 07:33

0

Its due to hash collision. Disabling dir_index resolved the issue.

answered Mar 29 '17 at 07:33

Wesley

855
1
9
23

Without dir_index, aren't there any issues with performance? Are file lookups still acceptably fast? – Seun Osewa Sep 23 '17 at 21:10

Python OSError No space left on device

2 Answers2