270

I am copying 7.5 GB file to a remote server using scp command. At some point in time file transfer breaks and I have to start all over again.

Is the temporary amount of file being transferred completely lost ? Can I somehow restart the transfer from where it has stopped at previous attempt ? If not, is there some standard Unix command line file transfer command for doing that ?

Darryl Hein
  • 142,451
  • 95
  • 218
  • 261
Matko
  • 3,386
  • 4
  • 21
  • 35
  • 3
    More answers found at [this question](http://superuser.com/q/421672/191090), incl. solutions without `rsync`. – tanius Jul 08 '16 at 19:52

2 Answers2

455

If you need to resume an scp transfer from local to remote, try with rsync:

rsync --partial --progress --rsh=ssh local_file user@host:remote_file

Short version, as pointed out by @aurelijus-rozenas: 

rsync -P -e ssh local_file user@host:remote_file

In general the order of args for rsync is

rsync [options] SRC DEST
Community
  • 1
  • 1
Tom McClure
  • 6,699
  • 1
  • 21
  • 21
  • 42
    For different port than default 22 use `rsync -P -e "ssh -p 222" SRC DEST` – asofyan Mar 09 '15 at 13:54
  • --rsh=ssh was unuseful for me (default) and I had to add -a or it would skip the incomplete file (assuming it was already transfered I guess) on OSX yosemite – Rayjax May 04 '15 at 14:35
  • 4
    And if the `rsync -e` is interrupted I found renaming the ..<6funnychars> to on destination before resuming the `rsync -e` was needed. – gaoithe Jul 28 '15 at 08:08
  • 5
    rsync only works if you have it available on the other side. – dothebart Jun 26 '16 at 09:33
  • What about the other way around? – Raffi Khatchadourian Oct 27 '16 at 21:19
  • 2
    Does rsync really resume a file transfer that was begun in scp? Are two rsync commands actually needed, one rsync, not scp, command to begin the copying, and then get disconnected, and then a second rsync command to resume the copying? – Geoffrey Anderson Nov 06 '16 at 14:36
  • 7
    It does not matter how you initially obtained the partial download. You may start out with scp and then finish with rsync. – Tom McClure Nov 07 '16 at 16:32
  • 1
    @TomMcClure I am finding that rsync is not working for this. I get that rsync completes the transfer but on the other side the file is only partially downloaded or gets replaced eventually. – mjs Mar 02 '17 at 07:49
  • It's also worth mentioning `rsync` works different with `/` at end of path. `rsync local_dir user@host:remote_dir/` will copy it like `remote_dir/local_dir/`, and `rsync local_dir/ user@host:remote_dir/` will copy all stuff from `local_dir` to `remote_dir`. – Qback Jan 23 '18 at 15:14
  • 1
    Will this solution also work when copying files from remote to local? I mean continue transaction when connection is back again. – abc Apr 13 '18 at 12:03
  • 1
    It's possible to interrupt rsync at any time with CTRL+c and repeat the command (rename local file etc). At the beginning rsync will check the partial file (if available) by checksumming (and fix it if broken by delta transfers). Then downloading (of the rest) of the file starts and the counter&progress&ETA are shown (that's for -P is for). Rsync if very efficient, I suggest to use rsync over ssh (scp is cp over ssh). Rsync has only client binary which could be used as a server, but with ssh the server does not need rsync listed on a port (ie no rsyncd service is needed on the remote computer) – Milan Kerslager Jul 28 '18 at 06:19
  • So does this not work for remote to local (I wonder since the answer emphasizes local to remote in bold). Or does it work the same? (I mean, is it just a matter of changing SRC and DEST? Or is there a fundamental difference?) – Kvothe Dec 07 '18 at 15:29
  • The `--partial` or `-P` transfers again whole file (!!!), see [this](https://unix.stackexchange.com/questions/48298/can-rsync-resume-after-being-interrupted) for full explanation and why you need `--append-verify` (I was hit by this in time presssure). Also rsync has serious performance hit in compare to scp. – Milan Kerslager Dec 09 '18 at 09:17
  • scp is faster because it does not do any compression or attributes transfer – Ciasto piekarz Feb 17 '19 at 14:55
  • 1
    Would that work if I was working recursively with scp -r instead of SCPing one large file? – Bluz Jun 12 '19 at 15:50
  • Check `ulimit -a | grep 'file size'` when it's failing with `File too large`. – Betlista Feb 15 '22 at 13:04
19

This is all you need.

 rsync -e ssh file host:/directory/.
DigitalRoss
  • 143,651
  • 25
  • 248
  • 329