1

I'm trying to create a sparse file (for a QEMU HDD image).

Both qemu-img and fallocate are proving confusing.

$ fallocate -l 100M disk.img

$ ls -lsh disk.img
101M -rw-r--r-- 1 i336 users 100M Jul 22 12:03 disk.img

Note the 101M. strace shows a successful syscall:

$ strace fallocate -l 100M disk.img
open("disk.img", O_RDWR|O_CREAT|O_LARGEFILE, 0666) = 3
fallocate(3, 0, 0, 104857600)           = 0

$ ls -lsh disk.img
101M -rw-r--r-- 1 i336 users 100M Jul 22 12:03 disk.img

I'm not sure if stat is the right tool, but just in case..

$ stat disk.img
  File: 'disk.img'
  Size: 104857600       Blocks: 204808     IO Block: 4096   regular file
Device: 802h/2050d      Inode: 549166      Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1337/    i336)   Gid: (  100/   users)

A possible (very weird) clue: 104857600/204808 = 511.9800. (File size / block count)

qemu-img has similar output. (I found the preallocation option in the manual.)

$ qemu-img create -f raw -o preallocation=falloc disk.img 100M
Formatting 'disk.img', fmt=raw size=104857600 preallocation=falloc

$ ls -lsh disk.img
101M -rw-r--r-- 1 i336 users 100M Jul 22 12:06 disk.img

Here's the annoying bit: the image appears to be using real space on disk.

$ df -h /; fallocate -l 1G disk.img; df -h /
Filesystem      Size  Used Avail Use% Mounted on
/dev/root        48G   43G  3.5G  93% /
Filesystem      Size  Used Avail Use% Mounted on
/dev/root        48G   44G  2.5G  95% /

And yet, just like a sparse file, it takes no time to create!

$ time fallocate -l 3.3G disk.img
0.00user 0.57system 0:00.91elapsed 63%CPU (0avgtext+0avgdata 5424maxresident)k
200inputs+0outputs (0major+68minor)pagefaults 0swaps

0.91 seconds, on a 5400RPM HDD. There is no way I'm not creating a sparse file.

And yet no matter what tool I use, it appears to be using 101MB of space right off the bat.

What could I be doing wrong or have misconfigured?

$ cat /etc/fstab
/dev/sda2 / ext4 rw,user_xattr 0 0
i336_
  • 1,813
  • 1
  • 20
  • 41
  • `fallocate` allocates data blocks without actually writing null zeros to those preallocated blocks. Basically those blocks are marked un-initialized and not all filesystems support such allocation. That's why `fallocate` could be fast. You've pre-allocated blocks in your command, that's what `stat` command tells you in its `Blocks` field. – GIZ Sep 04 '17 at 21:58
  • Oh, okay then. How do I determine how much space is actually being taken up on disk then? All documentation on the internet says that `df` reports physical usage (where sparse files do not count as physical usage). I'm not sure how to tell but I don't think I have sparse file support disabled (as I noted at the end I'm using ext4, which does state it supports sparse files). – i336_ Sep 05 '17 at 00:44
  • When you talk about physical usage, you should think about data blocks. Ask yourself how many data blocks does this file occupy? `df` cares about data blocks, then if your file occupy a data block, `df` has to count this block as occupied and thus not available. In fact, data blocks may have no data. If you want to check the physical size of a file use `ls -ls file` command, this gives you the size of the file on storage in terms of blocks. – GIZ Sep 05 '17 at 09:08
  • You can also use the classical `du` to see how much storage is actually used by the file. Sparse files will show a smaller number than the file size suggests. But AFAIU you have no sparse file created. You have a non-sparse file created which has lots of blocks which are marked as uninitialized internally; the file system will take care that their contents will appear to be zeroes. – Alfe Sep 08 '17 at 09:32

0 Answers0