I regularly need to transfer files that are a few gigabytes in size from a Linux VPS to my home Windows machine. I am trying to find a transfer method that fits my needs. FTP/SFTP works but is quite slow in comparison to other methods, SCP is fast enough but does not support resume which is important with files of this size, and rsync does not have a good Windows client and those that it does have are set up as backup solutions which are not exactly practical for this kind of use. Does anybody have any suggestions as to what I can use? Currently I am using SFTP but I can not bear the speed, it seems to be almost half of SCP for me.
2 Answers
From the man page for (rsync on my OpenBSD 4.9 box):
There are two different ways for rsync to contact a remote system:
using a remote-shell program as the transport (such as ssh or rsh) or
contacting an rsync daemon directly via TCP.
The point is I believe you can use rsync to connect using the first setup referred to above (remote-shell program as the transport) and get a broken connection resume for free by doing something like this:
$ rsync -r -h -P -e "ssh -p 1234" user@host:/my/huge/file anotherUser@anotherHost:/destination/
[Note on the switches: -r for recursive, -h for human-readable, -P for --partial (keep partially transferred files), and -e for specifying the remote shell to use. You can also use --partial-dir = DIR if you want to get fancy.]
and under this scenario you don't have to have an rsync server running on the windows box -- just an ssh server. There are many ssh servers to choose from that you can use on the windows platform (I suggest putty since this has the GUI setup familiar to win guys).
I'd also use md5 to hash your file and compare the transferred file's md5 to make sure your file is properly transferred and just keep firing off a script which keeps rsyncing from the linux side until you get your properly transferred file sitting pretty on the win box.

- 145
- 4
-
Looks good. I wasn't thinking about working in reverse and rsyncing from the server to my local computer. It would be nice if I didn't have to run SSH on my local computer, but if thats what it takes it will be fine. Thanks! – Ben Aug 18 '11 at 02:22