Downloading several files froma website with Perl

Question

I began to mess around with the Web in Perl. I use Windows, and ActivePerl. I wrote a script, which downloads mp3 files (It's an archive of a radio talkshow, all legal, in case you're wondering :) ) It (in theory)

parses a website
collects all links pointing to an mp3 file to an array
iterates through the array and downloads all files

However the script finds all the links, the array is OK, but downloads only one file. It must be a basic error, but I can't find it. Please point out my mistake, thank you.

use strict;
use warnings;
use WWW::Mechanize;
use LWP::Simple;
use File::Basename;

my $mech = WWW::Mechanize->new( autocheck => 1 );  
$mech->get("http://something_or_other.html");

my @mp3links = $mech->find_all_links(    
    tag       => "a",
    url_regex => qr/\.mp3$/,  
);  
print "\n";                 
foreach my $link (@mp3links) {  

    my $url       = $link->url();  
    my $localfile = basename($url);  
    my $localpath = "d:\\Downloads\\$localfile";

    print "$localfile\n";   
    getstore($url, $localpath);   
}

Have you tried using the debugger to check if @mp3links contains what you expect it to contain and that $localpath is generated correctly for each iteration? — pmakholm, Sep 18 '12 at 09:12
Does it work if you change `getstore()` call to `warn "$url => $localpath\n"`? Did you do `print Dumper(\@mp3links)` before entering `foreach` loop? — s0me0ne, Sep 18 '12 at 09:26
Does the `print "$localfile\n"` statement show all of the files you expect? — Borodin, Sep 18 '12 at 14:43
@mp3links contains the Mechanize::Link objects. $localpath is generated correctly in each iteration. I added the `warn` line as @s0me0ne suggested. Now the download works. I don't get it yet :). — librarian, Sep 18 '12 at 18:44

score 0 · Answer 1 · answered Sep 19 '12 at 23:43

Try this:

my $status = getstore($url, $localpath);
die "Error $status on $url" unless is_success($status);

Another possible culprit - if links contain query string (?-mark + params) basename will not chop it off. Problem is that this will make it invalid filename on windows.

score 0 · Accepted Answer · answered Sep 29 '12 at 18:33

0

Oh, sorry guys. This had nothing to do with Perl. My internet connection was rather sh***y, the connection was always lost ehile downloading the first file.

answered Sep 29 '12 at 18:33

librarian

129
10

Downloading several files froma website with Perl

2 Answers2