2

Problem: after enabling the ext4 large_dir option I want to know if it's used and/or whether I need to recreate directories for it to be used.

Detail: I have directories that on occasion can hold millions of files. I ran into:

[ma jan 11 01:37:22 2021] EXT4-fs warning (device nvme1n1): ext4_dx_add_entry:2349: Large directory feature is not enabled on this filesystem
[ma jan 11 08:42:21 2021] EXT4-fs warning (device nvme1n1): ext4_dx_add_entry:2345: Directory (ino: 11010158) index full, reach max htree level :2

So, I enabled large_dir:

tune2fs -O large_dir /dev/nvme2n1
tune2fs -O large_dir /dev/nvme1n1

Now, how can you tell it's used? lsattr -d shows nothing new:

# lsattr -d 0 1 2 ../corrupt_2012
-----------I--e--- 0
-----------I--e--- 1
-----------I--e--- 2
-----------I--e--- ../corrupt_2012   # Culprit with inode 11010158. It's been re-created after adding `large_dir`

The attributes I and e are 'index based on hashed trees' and 'extents'. Neither of those is new, and also appears on other disks/servers.

So, how do I know whether to recreate the directories?

Halfgaar
  • 8,084
  • 6
  • 45
  • 86

1 Answers1

2

tune2fs -l output will show the large_dir feature if enabled. Should be, was just set. I can find no other documentation on it, presumably proceed with creating more files.

From the inode count in the message, if most of those are in one directory the limit was 10 or 11 million. Which is consistent with coverage of largedir for Linux 4.13. Exceed the former limit, and the feature is in use.

As to integrity, I would hope the file system maintains integrity, by refusing more files if that index is full. But there might be a small chance of a problem, as this is not a common error condition. Consider running a fsck just to confirm consistency. And spot check the last file makes sense, examine metadata and contents.

John Mahowald
  • 32,050
  • 2
  • 19
  • 34
  • The contents of the dir in question is transient, and not _that_ important after some time has expired. To that end, I recreated the dir. I recreated all relevant dirs. Surprisingly, the two volumes I spread out IO over that were running 600 IO/s each, dropped to virtually nothing after enabling the feature. Utilization, IO/s, throughput, all very low. At least that suggests _something_ happened. – Halfgaar Feb 02 '21 at 21:43
  • What happened included you creating new files, reindexing the directory in a now 3 level HTree. Whether that made the difference or not, I am not sure. Proving it would likely require more testing, and why take the time if it performs better with the much higher limits. – John Mahowald Feb 03 '21 at 04:35