-1

I have a link.txt with multiple links for download,all are protected by the same username and password.

My intention is to download multiple files at the same time, if the file contains 5 links, to download all 5 files at the same time.

I've tried this, but without success.

cat links.txt | xargs -n 1 -P 5 wget --user user007 --password pass147
and
cat links.txt | xargs -n 1 -P 5 wget --user=user007 --password=pass147

give me this error:

Reusing existing connection to www.site.com HTTP request sent, awaiting response... 404 Not Found

This message appears in all the links i try to download, except for the last link in the file which starts to download.

i am currently use, but this download just one file at the time

wget -user=admin --password=145788s -i links.txt
Cesar
  • 9
  • 2
  • (1) `cat links.txt | xargs` would be better written `< links.txt xargs`. (2) You may want to quote the password if it contains characters which are special to the shell, such as `*`. (3) Try first to run `< links.txt xargs -n 1 -P 5 /bin/echo wget --user=user007 --password="pass147*"` to see exactly what commands is `xargs` trying to run. – AlexP Jul 16 '18 at 08:41
  • yes, I am currently using: `wget --user user007 --password "pass147*" -b -i links.txt`, the password does not contain any special characters at the moment. But the above command only downloads one file at a time Enter the command you gave me and throw me a lot of `wget --user=454545 --password=4546`, but don't run or download any. – Cesar Jul 16 '18 at 20:36
  • The point of the command I suggested was for you to see what *would* be executed by `xargs`. If you are satisfied that the commands to be executed then delete the `/bin/echo`. And see @VDR's answer -- it avoids the need for `xargs`. – AlexP Jul 16 '18 at 20:50

1 Answers1

1

Use wget's -i and -b flags.

-b
       --background
           Go to background immediately after startup.  If no output file is specified via the -o, output is redirected to wget-log.


-i file
       --input-file=file
           Read URLs from a local or external file.  If - is specified as file, URLs are read from the standard input.  (Use ./- to read from a file literally named -.)

Your command will look like: wget --user user007 --password "pass147*" -b -i links.txt

Note: You should always quote strings with special characters (eg: *).

VDR
  • 2,663
  • 1
  • 17
  • 13