1

I've been searching for the quickest way to copy some WAR and JAR files for huge software deployments between Windows systems. Clearly, multithreaded robocopy or any rsync protocol implementation will offer some increase in performance, but I'd like to know if there's a better option.

I am not looking for some GUI backup solution that mangles filenames or copies the files into incremental backup archives.

I'm interested if anyone has experience with copying across Windows network shares between domains via binary deltas or diffs. I've been looking at such technologies as DFS replication and rdiff-backup (which I believe lacks the ability to handle archives on Windows). I'd like to hear from others who have evaluated these types of technologies for copying between Windows systems. I need to learn which binary delta copy solution is the most viable and whether you would trust it for critical software deployments where every bit must match.

user95694
  • 111
  • 2

2 Answers2

0

Theoretically there is no assurance that you can transfer a message and it was not changed. No hashing/signing algorithm can offer 100% certainty that the data was not tempered. However, you can have a probability that is close to 100% that the data is intact if you use a good hashing algorithm.

So the larger the files are the more data you transfer, you have greater chances for some data to change without breaking the hash.

Practically you will see more problems from human mistakes - like programming errors, than changed files that have the same hash.

Mircea Vutcovici
  • 17,619
  • 4
  • 56
  • 83
  • That wasn't the question I asked. I'm specifically looking for copy mechanisms that will synchronize a binary archive between two Windows servers by transferring only the binary deltas. The hashing algorithm isn't the issue, unless we want to verify the data was copied accurately. I only want to know which binary delta-based copy mechanisms are reliable for Windows servers, if any. – user95694 Sep 23 '11 at 18:17
  • The integrity of data depends on the hashing algorithm used and, may be, on the software bugs. – Mircea Vutcovici Sep 23 '11 at 18:20
  • Well, it's more the software bugs that concern me. A lot of tribal knowledge around here says that Windows DFS replication used to be flaky in the 2000-2003 timeframe. In Windows Server 2003 R2, remote differential compression (RDC) was added to speed transfers. This satisfies our need for copying only the binary bits that changed in a large WAR archive, but I have no knowledge whether the service reliability issues were also fixed at that point. Clearly, choice of hashing technology has an impact, but it's not our highest concern. – user95694 Sep 23 '11 at 18:39
  • You will always face uncertainty when using computers. Even if they are supposed to be predictable. You just have to be prepared for the edge cases too. – Mircea Vutcovici Sep 23 '11 at 18:45
0

If it is about software deployment and you want to transfer binary deltas in a situation where you know the previous state, you might take a look at xdelta - it's not GUI and it's not a transfer protocol, but it does create binary deltas very efficiently and may be used in a software distribution scenario.

the-wabbit
  • 40,737
  • 13
  • 111
  • 174
  • This is helpful, but xdelta won't transfer binary deltas for compressed archives on Windows. We're mostly copying WARs and JARs, so it doesn't yet have the necessary functionality. We're now looking at a commercial product like Repliweb R-1. – user95694 Oct 03 '11 at 16:26
  • xdelta would create binary deltas for everything. Files compressed with an adaptive algorithm are usually not very good at being *deltaed*, but this is a different problem - mainly the one of the archiver. There are gzip patches to be "rsync-friendly" (i.e. give up some of the adaptive characteristics to improve the ability to get deltas), possibly there is something similar for ZIPs too. Without that, you would need to decompress before taking deltas / applying them - annoying, but doable, especially if you do not cryptosign your archives. – the-wabbit Oct 03 '11 at 21:18