We currently use rsync to backup virtual disks to another computer.
Rsync is slower than direct copy, but overwrite only modified parts in the destination files. This allow to make snapshots to have backup history, so we have several weeks of history with only 3.6 TB storage (source files are 1.6 TB).
Beside being slow, the main problem is rsync is often unable to do its job on big files, especially when they contains lots of identical data (ie zeros on the unused areas). It hangs forever.
We tested xdelta and xdelta3 but they are not efficient with big files.
We tested rdiff but this is not the right tool.
We tested open-vcdiff, also not the right tool.
We don't need a tool able to find similar data within the whole file. We only need to compare files block per block, and transfert them only when they differ.
Question: which tool can we use to compare 2 files block per block, and transfer the differences?
