0

Remote machine: a VPS running Debian 10; vsftp as ftp server.

Local machine: a Fedora 30 desktop, within my home LAN. Local router: a Technicolor AGHP, on lease from my telcom provider.

I am trying to download a 1.5G remote directory (~21.000 items) by

wget -m -c -N -X -v  --debug  -o wgout.txt  ftp://myuser:password@mydomain/html/wp/

Download starts fine, and progresses for almost 6/8 minutes, up to ~300MB, then it stalls:

250 Directory successfully changed.
done.
conaddr is: ip.ip.ip.ip
==> PASV ... 
--> PASV

227 Entering Passive Mode (ip,ip,ip,ip,234,149).
trying to connect to ip.ip.ip.ip port 60053
Closed fd 4
Closed fd 3
couldn't connect to ip.ip.ip.ip port 60053: Connection timed out
Retrying.

--2020-05-30 21:59:23--  ftp://myuser:*password*@mydomain/html/wp/wp-content/uploads/sites/3/2018/03/
  (try: 2) => ‘mydomain/html/wp/wp-content/uploads/sites/3/2018/03/.listing’
Found mydomain in host_name_addresses_map (0x55ccb875e0e0)
Connecting to mydomain (mydomain)|ip.ip.ip.ip|:21... Closed fd 3
failed: Connection timed out.
Releasing 0x00......0e0 (new refcount 1).
Releasing 0x00......0e0 (new refcount 0).
Deleting unused 0x000055ccb875e0e0.
Resolving mydomain (mydomain)... ip.ip.ip.ip
Caching mydomain => ip.ip.ip.ip
Connecting to mydomain (mydomain)|ip.ip.ip.ip|:21... Closed fd 3
failed: Connection timed out.
Releasing 0x00......10 (new refcount 1).
Retrying.

Afterwards, the remote ip is unreachable from any device within my LAN (either Linux, Win or Android), by any protocol (http(s), ssh, ftp), unless I reboot the router. (remote site is always reachable from outside my LAN.)

svftp.conf includes

connect_from_port_20=YES
pasv_enable=YES
pasv_min_port=1024
pasv_max_port=65535  (edited)

I am not even sure where to locate the issue: vsftp, router, local machine.

ps. is there a way to call openssh-sftp-server, i.e. something like:

    wget  **sftp**://myuser:password@mydomain/html/wp/
mario
  • 101
  • 2
  • there could be so many causes (especially the router failing with too many NAT states not yet garbage collected). Yet I notice that 65536 is an invalid port. Maybe use 65535 instead (or just don't set this parameter). – A.B May 30 '20 at 20:46
  • It stalls with pasv_min/max_port unset too. I was thinking about an issue with NAT table, but I do not know how I may check. Any hint? – mario May 30 '20 at 20:53
  • Is your public IP address changing when the router is rebooted? – A.B May 30 '20 at 20:55
  • yes, it does so, why? – mario May 30 '20 at 21:00
  • If it didn't I would have blamed the router, but now it's also possible to blame the vsftpd's side, like a security kicking in etc. So you should consult the vsftpd's logs and its global system logs to see if any security was tripped and disabled temporarily your IP address – A.B May 31 '20 at 11:12
  • nope, vsftpd.log shows just lines as `Sun May 31 16:27:28 2020 [pid 4221] [myuser] OK DOWNLOAD: Client "::ffff:ip.ip.ip.ip", "/var/www/html/wp/wp-content/themes/parabola/fonts/elusive.svg", 28341 bytes, 539.35Kbyte/sec` – mario May 31 '20 at 14:29
  • out of ideas sorry. Maybe you'll discover new evidences later. You should change the download method (ftp is insecure etc etc anyway). eg SFTP – A.B May 31 '20 at 14:31
  • I am trying rsync – mario May 31 '20 at 15:14

1 Answers1

0

If this is a problem of too many connections per time period then you would want to limit wget to a certain amount. It does not seem to offer that feature, though. But you could use --limit-rate to slow down the transfer as a whole. If it is slow enough the problem may disappear.

Or you change your download so that it does not create a huge amount of connections. One way is to use something better than FTP as you did with rsync. You did not mention whether there are strong reasons why you prefer wget over rsync.

Unfortunately wget does not support SFTP but curl does (and may be closer to what you want than rsync.

If for some reason you really prefer wget over anything else then you could create an SSH tunnel. No port forwarding (as that would not help with FTP) but creating virtual network interfaces on both sides:

ssh -o 'Tunnel point-to-point' ...

wget would connect to the remote tun IP address. For all systems in between it would look like one long connection. You could even easily use traffic shaping that way to prevent those transfers from affecting the rest of the systems.

Hauke Laging
  • 5,285
  • 2
  • 24
  • 40