I'm trying to transfer a huge growing logfile in a way that will survive network partitions between the client and server. If the network connection is lost, I cannot afford to retransmit the entire file because of gross network utilization restraints. I will only sync an ongoing compressed version as follows
tail -f logfile | xz -3 -c > syncThis.xz
I've found multiple options for conducting the transfer on a continuous basis.
The one that I can understand best is rsync --append. If the connection cuts out, rsync will fail, then I can rerun the command to resume the transfer.
Another option seems to be sshfs or nfs, with or without autofs. With one of these the command above would change to
tail -f logfile | xz -3 -c > /mountLoc/syncThis.xz
and there would be no need for rsync.
The question is this: With a networked filesystem, if I continue appending to a synchronized file and the network cuts out and then resumes, will only and all of the remaining appends of the file be automatically transferred, similar to what rsync --append would do?
If this is not the default, is there a setting to enable such a thing?
I suppose such a filesystem would have to keep track of the fact that only appends had occurred on the file in the interim.
