3

I have a few PC's with an ext4 filesystem that I want to backup to a file server which is also ext4. Problem is there are some discrepancies in file sizes when using rsync, and I have noticed this is due to sparse files.

The problem is I want to create an exact rsync copy of the filesystem using rsync over a network to keep weekly backups, in case I need to restore, and the restored data should be the same size as whats running on the PC.

Creating the test files, 1 sparse and 1 not:

mkdir testing
dd if=/dev/zero of=testing/sparse-file.img bs=1 count=0 seek=5M
cp testing/sparse-file.img testing/non-sparse-file.img --sparse=never

Rsync with and without sparse option:

mkdir testa testb
rsync testing/* testa
rsync --sparse testing/* testb

Results:

du -h
5.1M    ./testing
4.0K    ./testb
11M     ./testa
16M     .

testing has 1 file 5MB and one sparse file, testb had both files become sparse, testa had both files become non-sparse

But how do I make rsync maintain the file sparseness? So the filesystem will have the exact same size on the restored system.

I want to have certainty when I restore my system I'll know exactly how big the restored data will be, with sparse option, my restored system is going to be more sparse then what it was originally (I guess this is acceptable), and with the non-sparse option, this will result in an unpredictable larger restored system.

Brandon
  • 31
  • 1
  • @Colt that only explains the size discrepancies may be due to spare files, but doesn't come up with a solution – Brandon Sep 15 '18 at 04:24
  • Did you look at [this answer](https://stackoverflow.com/questions/19821186/rsync-sparse-does-transfer-whole-data#19827546) – danblack Sep 15 '18 at 04:48
  • This does not look like a duplicate to me. The older question is asked by somebody who appears to not have known what sparse files is, and the accepted answer explains sparse files. This question however demonstrates an understanding of sparse files, and explicitly tested both with and without the `--sparse` option. Effectively what this question is asking for an `rsync` equivalent of `cp --sparse=auto`, which is not at all addressed by the older question. – kasperd Sep 17 '18 at 08:14
  • @kasperd The third answer (https://serverfault.com/a/749584/120438) answers this question too, so I think it would be appropriate to keept this question closeda as a duplicate. – Jenny D Sep 19 '18 at 09:18
  • @JennyD Actually it doesn't. That answer mentions `--sparse`, but this question mentions that `rsync` has been tried both with and without `--sparse`, and neither produced the desired result. – kasperd Sep 19 '18 at 13:25
  • 1
    @kasperd I agree, I actually saw that question before I posted mine, maybe there isn't a way in rsync to do it, womble could of at least provided a comment explaining how this is a duplicate before closing this :/ – Brandon Sep 20 '18 at 13:35
  • 1
    @womble If there is no solution, then the duplicate thread doesn't say that, and if there is a solution there, it obviously isn't the answer to my question. – Brandon Sep 20 '18 at 13:36

1 Answers1

1

I think you perceive there is a problem when there is none.

If you have many sparse files, then it would be obviously bad if the restore loses the sparseness and causes your disk to be full.

But if an original file was not sparse and the restored file is sparse, there is no problem. The blocks missing in a sparse file return zero on read. The files that were originally not spares contains large enough blocks of zeros that are sparse blocks in the copy. To any application reading the file, the result es exactly the same. Except the reading the sparse blocks is also faster, because the memory is just filled with zero instead of reading it from the disks. So you can consider sparse files both an optimization of disk space and access time. You could even regularly check your files and try to convert them to sparse files if you think it's worth.

For a long time there was no way to determine if a block of a file is allocated on disk or not. Recently some Linux file systems have support to find sparse blocks in a file. If your applications really depend on the sparseness information, you can extract that to a different file, include that in the backup, and restore that sparseness later.

But most applications that create sparse files don't care about the content of the sparse blocks. The blocks have never been written, or they would not be sparse. The application knows not to expect data in these blocks.

So why exactly do you think this should be a problem?

RalfFriedl
  • 3,108
  • 4
  • 13
  • 17
  • There are a few cases where you want a file to not be sparse. Swap files on Linux must not be sparse. And sometimes you may create files in which you later intend to do random access writes which you do not want to fail due to the disk running out of space, which could happen if the file is sparse. – kasperd Sep 17 '18 at 08:08