Stack Overflow
Stack Overflow

Questions tagged [www-mechanize]

WWW::Mechanize is a Perl module for programmatic web browsing, used for automating interaction with websites.

Features include:

  • All HTTP methods
  • High-level hyperlink and HTML form support, without having to parse HTML yourself
  • SSL support
  • Automatic cookies
  • Custom HTTP headers
  • Automatic handling of redirections
  • Proxies
  • HTTP authentication

Mech supports performing a sequence of page fetches including following links and submitting forms. Each fetched page is parsed and its links and forms are extracted. A link or a form can be selected, form fields can be filled and the next page can be fetched. Mech also stores a history of the URLs you've visited, which can be queried and revisited.

See the home page for complete details.

366 questions
5
votes
1 answer

Logging into a website using Mechanize and Nokogiri?

I am having some difficulty with one of our service providers login forms. The other sites are working fine but for some reason I can't get past their login form. The website login for is like this:
dannymcc
  • 3,744
  • 12
  • 52
  • 85
5
votes
5 answers

How do I use and debug WWW::Mechanize?

I am very new to Perl and i am learning on the fly while i try to automate some projects for work. So far its has been a lot of fun. I am working on generating a report for a customer. I can get this report from a web page i can access. First i will…
user119652
5
votes
3 answers

Transparently Handling GZip Encoded content with WWW::Mechanize

I am using WWW::Mechanize and currently handling HTTP responses with the 'Content-Encoding: gzip' header in my code by first checking the response headers and then using IO::Uncompress::Gunzip to get the uncompressed content. However I would like…
Gurunandan Bhat
  • 3,544
  • 3
  • 31
  • 43
5
votes
1 answer

Perl WWW::Mechanize: How can I specify the destination IP address independently of the URL?

I want to use Perl www::mechanize to connect to the webserver and request a resource. E.g. http://www.my.domain/test.html. But I want to specify the IP address independently from the hostname in the URL. For example: www.my.domain resolves to…
Joerg
  • 81
  • 3
5
votes
3 answers

How can I download link targets from a web site using Perl?

I just made a script to grab links from a website, and in turn saves them into a text file. Now I'm working on my regexes so it will grab links which contains php?dl= in the url from the text file: E.g.:…
eraldcoil
  • 55
  • 1
  • 4
5
votes
4 answers

How can I get WWW-Mechanize to login to Wells Fargo's website?

I am trying to use Perl's WWW::Mechanize to login to my bank and pull transaction information. After logging in through a browser to my bank (Wells Fargo), it briefly displays a temporary web page saying something along the lines of "please wait…
J Miller
  • 51
  • 1
  • 2
5
votes
1 answer

How to do asynchronous www-mechanize using anyevent

I've been doing a fair amount of research on the topic and while there are some questions out there that relate, I'm really having a hard time understanding how to properly do async programming using AnyEvent and www-mechanize. I'm trying to stick…
mikew
  • 912
  • 2
  • 11
  • 22
5
votes
3 answers

How do I download a file with WWW::Mechanize after it submits a form?

I have the code: #!/usr/bin/perl use strict; use WWW::Mechanize; my $url = 'http://divxsubtitles.net/page_subtitleinformation.php?ID=111292'; my $m = WWW::Mechanize->new(autocheck => 1); $m->get($url); $m->form_number(2); $m->click(); my $response…
codygman
  • 832
  • 1
  • 13
  • 30
5
votes
4 answers

Perl - Regex to extract only the comma-separated strings

I have a question I am hoping someone could help with... I have a variable that contains the content from a webpage (scraped using WWW::Mechanize). The variable contains data such as these: $var = "ewrfs sdfdsf…
yonetpkbji
  • 1,019
  • 2
  • 21
  • 35
5
votes
3 answers

How do I use Perl's LWP to log in to a web application?

I would like to write a script to login to a web application and then move to other parts of the application: use HTTP::Request::Common qw(POST); use LWP::UserAgent; use Data::Dumper; $ua = LWP::UserAgent->new(keep_alive=>1); my $req = POST…
maxjackie
  • 22,386
  • 6
  • 29
  • 37
4
votes
2 answers

Can't create an instance of WWW:Mechanize::Firefox

I've just installed WWW:Mechanize::Firefox, but when I tried to create an instance of this "class": my $mech = WWW::Mechanize::Firefox->new(); I get this error: Failed to connect to , problem connecting to "localhost", port 4242: No…
eliane
  • 95
  • 1
  • 7
4
votes
1 answer

Memory leak using WWW::Mechanize

I have this script in Perl, and it is getting a "Out of memory" error after a few minutes of running. I can't see any circular references and I can't work out why it is happening. use feature 'say'; use WWW::Mechanize; use…
CJ7
  • 22,579
  • 65
  • 193
  • 321
4
votes
2 answers

perl WWW::Mechanize, link redirect problem

I use WWW::Mechanize::Shell to test stuff. my code is this: #!/usr/bin/perl use WWW::Mechanize; use HTTP::Cookies; my $url = "http://mysite/app/login.jsp"; my $username = "username"; my $password = "asdfasdf"; my $mech =…
Sanx
  • 223
  • 1
  • 6
  • 17
4
votes
1 answer

Trouble With CPAN Module

I've tried to install the WWW::Mechanize module with 'cpan WWW::Mechanize' I get no errors on the 'use WWW::Mechanize' line which means its finding the files, but upon trying to instantiate it with: $m = WWW::Mechanize->new(); I get the following…
Arjun Nayini
  • 191
  • 1
  • 2
  • 5
4
votes
1 answer

Use perl WWW::Mechanize on a local file

I'm currently working on a Perl script and I use the CPAN module WWW:Mechanize to get HTML pages from websites. However, I would like to be able to work on offline HTML files as well (that I would save myself beforehand most likely) so I don't need…
Azaghal
  • 430
  • 4
  • 12
1
2
3
…
24 25