3

I'm writing 100 MB files to a disk with ext4 filesystem continually, and when the disk is full, I just delete the oldest file and write the new one. The files never grow in side. When I do this the file system becomes quiet fragmented after some time. I've looked into all formatting options for ext4, but haven't figured out why such fragmentation happens.

Basically, to simplify the matter, what I do is the following in a loop:

int32_t fid = open(filepath, O_RDWR | O_CREAT, S_IRUSR | S_IWUSR);
int success = fallocate(fid, 0, 0, MAX_FILE_SIZE);
close(fid);

Is there a way to tell the file system to write the data to disk in contiguous blocks? I've looked at other posts, but I haven't quite figured it out. Should I be using malloc() instead fallocate()?

Nathan Ernst
  • 4,540
  • 25
  • 38
user3255479
  • 31
  • 2
  • I don't think it should be tagged c++ – Joky Jan 31 '14 at 01:00
  • Just a guess, but maybe the fragmentation is because space allocated for directories (or unrelated activity like log files getting appended) is getting in between your files? If you free a 100MB file and one of those blocks gets used for something else, then the 100MB file you write immediately afterwards, thinking you're just replacing it, will in fact be fragmented. That's the thin end of the wedge for your beautiful contiguous disk. Or if your free space is not a multiple of 100MB, then when you delete a file there's no guarantee the next file will use its space first, before the stub-end. – Steve Jessop Jan 31 '14 at 01:23
  • Actually, I always delete old files so that the amount of free disk space is 3 times the file size (i.e. at least 300MB of free space in my case). I was hoping this would allow room for at least one contiguous range for the new file. Even when I set the free disk space to larger, i.e. 1GB, the fragmentation results after sometime. It looks like fallocate does not allocate contiguous disk space even when its available. Is that correct? – user3255479 Jan 31 '14 at 17:57

1 Answers1

0

Ext4 has an optional support of "extents", that are contiguous blocks of disk. The support must be enabled with tune2fs. Here some useful links:

https://serverfault.com/questions/400026/linux-ext4-extents-attribute

https://www.kernel.org/doc/Documentation/filesystems/ext4.txt

http://ext2read.blogspot.it/

Community
  • 1
  • 1
Giuseppe Guerrini
  • 4,274
  • 17
  • 32