Diff and patch for big binary files

Question

As part of my job I regularly need to upload big files (a 512 MB to 1 GB ISO, for example). In some cases, very little changes from ISO to ISO (say, a 10 MB file in a 1 GB ISO). Unfortunately, I can't just send the changed file - I need to send the whole ISO. A faster connection is not a valid solution.

Is there any tool (preferably Linux, preferably free) that can make and apply diffs of such big binary files?

Can't you build the .iso on the remote server? update the file remotely and then master the iso on the remote server. — BuildTheRobots, Jan 14 '10 at 23:13
An additional requirement would be that it also has a Windows version. Looks like xdelta could be what I'm looking for. — ggambetta, Jan 14 '10 at 23:14

score 6 · Accepted Answer · answered Jan 14 '10 at 23:19

6

I use xdelta for this purpose, others use rdiff. Both work nicely.

answered Jan 14 '10 at 23:19

womble

96,255
29
175
230

Can you provide a link please? – not2qubit Oct 23 '16 at 11:48
1

don't try it if you can not search it – Behnam Sep 26 '17 at 17:03
Use xdelta3 which is way better than xdelta. – Smeterlink Dec 22 '20 at 20:09

score 4 · Answer 2 · answered Jan 25 '13 at 02:12

4

BSDiff is more efficient.

http://www.daemonology.net/bsdiff/ "bsdiff and bspatch are tools for building and applying patches to binary files. By using suffix sorting (specifically, Larsson and Sadakane's qsufsort) and taking advantage of how executable files change, bsdiff routinely produces binary patches 50-80% smaller than those produced by Xdelta, and 15% smaller than those produced by .RTPatch (a $2750/seat commercial patch tool). "

answered Jan 25 '13 at 02:12

jwilkins

511
3
4

3

While a nice tool, please note the running time and complexity: "`max(17*n,9*n+m)+O(1)` bytes of memory, where n is the size of the old file and m is the size of the new file." That's a minimum of 17 times the size of the old file. I came here looking for an alternative because trying to come up with a diff of my 1.56 GB ISO took a bit too long (I gave up waiting) with a quad core cpu and 16gb of memory. – vmrob Dec 10 '14 at 01:00

score 1 · Answer 3 · answered Jan 14 '10 at 23:35

1

We use rdiff for replicating database snapshots to a backup server at remote offices every hour. It's been working pretty well for several months now.

All your need is rdiff and cygwin1.dll and you're good to go.

answered Jan 14 '10 at 23:35

Matt

142
1
8

score 1 · Answer 4 · answered Jan 14 '10 at 23:43

1

Is it possible to rsync the files? This will find an apply small binary differences.

answered Jan 14 '10 at 23:43

brianegge

1,064
2
14
23

Diff and patch for big binary files

4 Answers4