-1

On a big corporate firewall-ridden network, we have traditionally used scp to copy a few brand new files over from one machine to another, as part of the log-file rotation and analysis routine.

However, it has recently been observed that the scp seems to fail randomly, with the Connection reset by peer error message, and our routines break.

Would it be a good idea to switch to rsync in place of scp, or is there a way to make scp a little bit more persistent in trying to copy the files over?

HopelessN00b
  • 53,795
  • 33
  • 135
  • 209
cnst
  • 13,848
  • 9
  • 54
  • 76
  • Both are based on ssh and connection reset is an event occuring at the TCP layer likely to be sent by intermediate firewalls. Get in touch with your net team to properly configure SSH `(Server|Client)AliveInterval` and `(Client|Server)AliveMaxCount` regarding corporate firewall timeouts (also make sure you don't hit some potential burst rules limit set by your net team). – Xavier Lucas Sep 20 '14 at 22:27
  • @XavierLucas, the alive interval is probably only meaningful for stale connections and ssh itself; I think scp establishes a new connection for every transfer, so, I'm not sure that `(Server|Client)AliveInterval` come into play here -- would it even become activated if the files are being actively transferred over the connection? – cnst Sep 21 '14 at 15:43
  • Yes, scp only uses one SSH tunnel to transfer multiple files as it is sequential. These options are base SSH options used by scp and any tool using SSH generally speaking, so it comes into play any time SSH is used. To address the reconnecting part, use `ConnectTimeout` and `ConnectionAttempts`. – Xavier Lucas Sep 21 '14 at 17:24

1 Answers1

3

"Connection reset by peer" means that the remote end of the TCP stream (not the end which logged the error) closed the TCP stream unexpectedly. This can happen if the program on the remote end has crashed or been killed, for example. The error can also be induced by some network devices--like stateful firewalls or NAT devices--interfering with the TCP stream.

In the best case, whatever is causing the error could be specific to the scp program. Rsync doesn't use scp so it might be unaffected. Alternately, the problem could be generic to ssh connections between the two servers, or even to any TCP stream between the two servers. You'd probably be running rsync through ssh, so it might experience the same problem you're having with scp.

One of the main features of rsync is that, when a destination file already exists but has different contents than the source, rsync doesn't need to retransmit the entire file--it just needs to transmit the additional data to make the destination match the source. When you run rsync and it fails before it finishes, you can restart rsync and it'll pick up where it left off, rather than start over. If you find that rsync is crashing on you like scp has been, then this rsync feature will be extremely valuable.

Rsync will not actually try to reconnect on its own, but you can keep running rsync until it reports success. For example, this bash shell command would do it:

until rsync ...
do
    echo Retrying rsync
done
Kenster
  • 2,152
  • 16
  • 16