1

I have a large hard disk and a second hard disk, that I use as backup drive (with an USB 3 adapter), using rsync:

rsync -ahHPxX --delete --delete-delay

Rsync "pauses" a long time during sync, while I can hear both hard disks seeking. I think when I used XFS on both drives, it was a lot faster. I am not sure, because I cannot compare it directly.

Is rsync on btrfs slow for some reason? Maybe it is copying all metadata, or writing access times, that were not written when I used XFS?

I sync from a read-only snapshot on the first drive directly on the second drive (without using a second subvolume for it).

allo
  • 1,620
  • 2
  • 22
  • 39
  • Why do you want to use rsync, when you just can use btrfs-send instead? – Marc Stürmer Oct 08 '19 at 20:45
  • @MarcStürmer Because they do different things. They may look similar, but they are very different. Both are useful, but you cannot replace rsync with btrfs-send for most purposes. – allo Oct 09 '19 at 21:08

1 Answers1

2

From my experience, BTRFS fragments horribly and this can have devastating performance effects on rotating media (HDD). This is amplified by rsync as it copies modified files by creating a temporary file which can be written in very small pieces (due to how the delta alg works).

XFS, on the other hand, is an extremely fast and optimized filesystem; recent (post-2015) versions also provide metadata checksum (see mkfs.xfs -m crc=1), albeit lacking data checksums and filesystem-level snapshots.

In short: never expect high performance by BTRFS, or you will be disappointed. It has many interesting features, but it is much slower than XFS (or even EXT4).

shodanshok
  • 47,711
  • 7
  • 111
  • 180
  • I expected something like this. I cannot easily defragment the live drive, as it uses snapshots, that would break and consume a lot more space. Do you think reformatting the backup drive would help or is reading the files the bigger problem than writing the backup? I cannot "just try it" without syncing for a long time, because both drives are 8 TB. – allo Sep 30 '19 at 21:25
  • 1
    @allo you should run `iostat -x -k 1` during a transfer to understand which drive is the bottleneck (ie: reading from source disk vs writing to destination disk). If possible, post (an extract of) `iostat` output. Do *not* reformat anything without performance data. – shodanshok Sep 30 '19 at 21:51