I am currently using wget -m to mirror a remote FTP but the speed suck, what are the good alternative (command) so we can parallel to download in order to boost the speed up?
Asked
Active
Viewed 2.4k times
9
Ryan
- 5,831
- 24
- 72
- 91
-
1Do you have any control over the remote FTP? Or does it offer ``rsync``? That would be much more efficient depending on the data set you like to mirror. – gertvdijk Nov 21 '12 at 11:07
2 Answers
20
As mentioned by pQd lftp is a great tool for that and it's included in "all" distributions.
lftp -u user,pwd -e "mirror --parallel=3 --verbose /$REMOTE_DIR /$LOCAL_DIR" $REMOTE_HOST
You can run it several times with time and different values of --parallel to find the best value.
Pablo Martinez
- 2,406
- 17
- 13
-
1Worth adding here that `mirror -R` will mirror the local directory on the ftp server. – LondonRob May 11 '18 at 13:52
10
Take a look at lftp. It's quite a powerful ftp client that has an option for parallel downloads:
'mirror' can download several files in parallel (--parallel option) and a single file with 'pget' (--use-pget-n option).
It's available in standard debian's repository so you should have it as well in ubuntu without using 3rd party binaries or compiling it from source.
Andrew Schulman
- 8,811
- 21
- 32
- 47
pQd
- 29,981
- 6
- 66
- 109