0

I have a question about the filesystem ext4 and disk fragmentation: Does ext4 place files in the same directory near or next to each other on the disk? And if so how is it fragmented, for example is filename important?

I ask this because we've a project for which we have a lot of small files and the files are read/requested in multiple threads but every thread has it's patern for file requests. If we place files in the same directory (file001, file002, file003 etc.) and give the disk a "Read Ahead" for 1 or 2 MB it will boost the read speed.

The idea is o have the files in the same directory next to each other on the disk. (file001, file002, file003 etc.)

klaasio
  • 79
  • 5
  • Save development time (cost) and buy SSD to overcome your issue. – Alex Berry Jul 28 '14 at 15:59
  • You also may want to look at other file systems. There are file systems better optimized for many small files. Or at least, tune your ext4. But as far as fragmentation, the directory does not influence where they are stored, as far as I know. It just tries to put files in empty places throughout the file system. – Halfgaar Jul 28 '14 at 16:29
  • @Halfgaar perhaps you know a way to "defragment" directories, so that files in an directory are stored "next" to each other on the disk so we can boost "sequantial" reading and add readahead. – klaasio Aug 01 '14 at 21:59
  • @AlexBerry thx for you thought, but we need a lot of storage (+400TB), a possible solution would be that we read all the files that we expect to be requested from the hdd to an SSD disk. But still the files should then be read sequential from the hdd so they can be transfered fast enough. – klaasio Aug 01 '14 at 22:04
  • For example i found the tool: Shake: http://www.vleu.net/shake/ – klaasio Aug 01 '14 at 22:13

0 Answers0