Stack Overflow
Stack Overflow

Questions tagged [www-mechanize]

WWW::Mechanize is a Perl module for programmatic web browsing, used for automating interaction with websites.

Features include:

  • All HTTP methods
  • High-level hyperlink and HTML form support, without having to parse HTML yourself
  • SSL support
  • Automatic cookies
  • Custom HTTP headers
  • Automatic handling of redirections
  • Proxies
  • HTTP authentication

Mech supports performing a sequence of page fetches including following links and submitting forms. Each fetched page is parsed and its links and forms are extracted. A link or a form can be selected, form fields can be filled and the next page can be fetched. Mech also stores a history of the URLs you've visited, which can be queried and revisited.

See the home page for complete details.

366 questions
0
votes
1 answer

Regex in WWW::Mechanize in Perl

I am not sure what is the correct syntax for the url_regex used in WWW::Mechanize. I am collecting all the links from a web page that start with an http:// and they are of the following format: http://google.com and…
Neon Flash
  • 3,113
  • 12
  • 58
  • 96
0
votes
1 answer

Filling up form using perl WWW::Mechanize::Formfiller

I wish to download information from the website http://revigo.irb.hr/ which have an interactive menu. I'm trying the following Perl code after fetching out the form fields using perl -MWWW::Mechanize::Shell -e shell get…
WoA
  • 173
  • 2
  • 2
  • 12
0
votes
2 answers

perl get ip of website using lwp or www::mechanize

So I am working on a crawler, and some of the data I want to store about the sites I crawl is their IP address. I'd prefer to do this without having to hit their server again, so is there anyway to get this information from LWP or WWW::Mechanize…
srchulo
  • 5,143
  • 4
  • 43
  • 72
0
votes
1 answer

How to use output from WWW::Mechanize?

I would like to loop through all links on a web page, so I have tried #!/usr/bin/perl use WWW::Mechanize; my $url = "http://www.google.com"; my $m = WWW::Mechanize->new(); $m->get($url); my @links = $m->find_all_links(url_regex =>…
Sandra Schlichting
  • 25,050
  • 33
  • 110
  • 162
0
votes
1 answer

How to manipulate a DOM element while running a perl mechanize script?

This post discusses how to inform the user via a progress bar by calculating the percentage of the file being downloaded by a www::mechanize Perl script. I need to inform my user of a www::mechanize script's progress but not necessarily in…
Ricalsin
  • 950
  • 9
  • 28
-1
votes
2 answers

Absolutely NO perl modules?

I'm trying to install WWW::Mechanize; but of course, something had to go wrong. I purchased a VPS, and whenever I load up CPAN (AKA The quick way), it says I'm missing modules. Such as... HTTP, LWP, Test, HTML, LOADS of things that I thought would…
Saustin
  • 1,117
  • 6
  • 15
  • 27
-1
votes
1 answer

perl WWW::Mechanize follow link in a certain table row

I'm trying to write a script, that transfers internal office mails to my normal mail-account. The only way I see to identify unread mails is, that they are in a with a special style ('bold preview'). I'm able to store the -element in a…
Franz Müller
  • 21
  • 1
-1
votes
1 answer

How to find a word beginning with specified letters then remove everything else in the string?

my @buildno = $mech->xpath('/html/body/form/table/tbody/tr[2]/td[3]/table /tbody/tr[2]/td/table/tbody/tr[1]/td/table/tbody/tr[1]/td', type => $mech->xpathResult('STRING_TYPE')); I have the above code which contains a string. I need to capture the…
Jamal Mahroof
  • 33
  • 8
-1
votes
1 answer

Perl - geturls with WWW::Mechanize

I am trying to submit a form on http://bioinfo.noble.org/TrSSP/ and want to extract the result. My query data looks like this >ATCG00270 MTIALGKFTKDEKDLFDIMDDWLRRDRFVFVGWSGLLLFPCAYFALGGWFTGTTFVTSWYTHGLASSYLEGCNFLTAA …
pali
  • 195
  • 2
  • 14
-1
votes
1 answer

How do you remove indentation from a single element in an array - perl?

@para_text[2] = $mech->xpath('/html/body/form/table/tbody/tr[2]/td[3]/table/tbody/tr[2]/td/table/tbody/tr[1]/td/table/tbody/tr[4]', type => $mech->xpathResult('STRING_TYPE')); I have the above code which returns the following in this format: …
Jamal Mahroof
  • 33
  • 8
-1
votes
1 answer

WWW::Mechanize: Unable to find Form

I'm trying to automate logging in to a website. The website is https://www.remind.com/log_in. Using dump_forms gives no forms on the page. I tried using this to submit the form anyway, but this fails to compile: use strict; use warnings; use…
Aditya J.
  • 131
  • 2
  • 11
-1
votes
2 answers

Why doesn't my WWW::Mechanize submit a PayPal form work?

Nothing I've tried can get my code to submit correctly. Can anyone else figure this out? #!/usr/bin/perl use WWW::Mechanize; my $user = 'cowsaremyfriends@gmail.com'; my $pass = 'hackswipe'; # Test account; don't worry my $browser =…
Ryan Lester
  • 2,363
  • 7
  • 25
  • 38
-1
votes
1 answer

https and WWW::Mechanize - accept specific certificate (MD5/SHA1/...)

How to make WWW::Mechanize accept specific SSL certificate for https server? [SSL certificate should be validated by MD5/SHA1/... checksum ] Background: So far I need a hack to accept expired (a few days so far) SSL certificate [replace standard set…
AnFi
  • 10,493
  • 3
  • 23
  • 47
-1
votes
3 answers

Why can't Perl's WWW::Mechanize find the form by field names?

#!/usr/bin/perl use WWW::Mechanize; use Compress::Zlib; my $mech = WWW::Mechanize->new(); my $username = ""; #fill in username here my $keyword = ""; #fill in password here my $mobile = $ARGV[0]; my $text = $ARGV[1]; $deb = 1; print…
Anil
  • 3,912
  • 5
  • 35
  • 46
-1
votes
1 answer

Bypass asp login with Mechanize

I have some problems with login in asp page. Here is my code: #!/usr/bin/perl -w use strict; use warnings; use WWW::Mechanize; use LWP::Debug qw(+); my $mech = WWW::Mechanize->new; $mech->agent('User-Agent=Mozilla/4.0'); $mech->get(…
John Smith
  • 47
  • 8
1 2 3
…
24
25