svnsync cannot recover from source idle disconnect

Question

If I try to mirror a 3rd-party repo to a local one then while svnsync is uploading a 300M+ file to the slow destination server the source server disconnects and svnsync does not try to reconnect to it and enters infinite loop. How to workaround this?

C:\progs\svnsync-dir>svnsync --version
svnsync, version 1.14.0 (r1876290)
   compiled May 24 2020, 17:07:49 on x86-microsoft-windows

C:\progs\svnsync-dir>svnsync --steal-lock sync https://targetreposito.acme.com/s
vn/Reponame
Stole lock previously held by 'my-host-name:ab204a7a-756f-2947-af9c-caf904a8945d
'
Transmitting file data .........................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
............< here it hangs

basin · Accepted Answer · 2023-02-07T11:11:16.843

After compiling the svn communication library serf from trunk svnsync instead of hanging is now crashing with An error occurred during decompression or (if

[global]
http-compression = no

) in ~/.subversion/servers then with Malformed XML: no element found.

After dumping the HTTP conversation between svnsync and the source server I figured out that the source server closed connection while svnsync was flushing partially received delta to the destination server.

The solution is to use the pv tool with slice disabled and a huge buffer to hold the full response: pv -q -C -B4G

~~Add to /etc/hosts:~~

127.0.0.3 svn.acme.com
1.2.3.4   svn.acme.com.real

Then run:

sudo socat TCP-LISTEN:443,reuseaddr,fork EXEC:"./hugebuf.sh svn.acme.com.real 443",nofork

Run:

socat TCP-LISTEN:3128,reuseaddr,fork EXEC:"./hugebuf.sh -p",nofork

hugebuf.sh (simplified):

#!/bin/bash
host=${1:?}
port=${2:?}
exec 4>"/dev/tcp/${host:?}/${port}"
cat <&0 >&4 &
pv -q -C -B4G <&4

Lastly, run the svnsync command with extra args:

svnsync --config-option servers:global:http-proxy-host=localhost --config-option servers:global:http-proxy-port=3128

Now it's able to commit this very revision.

The above works with msys / git-for-windows too.

svnsync cannot recover from source idle disconnect

1 Answers1