Stack Overflow
Stack Overflow

Questions tagged [www-mechanize]

WWW::Mechanize is a Perl module for programmatic web browsing, used for automating interaction with websites.

Features include:

  • All HTTP methods
  • High-level hyperlink and HTML form support, without having to parse HTML yourself
  • SSL support
  • Automatic cookies
  • Custom HTTP headers
  • Automatic handling of redirections
  • Proxies
  • HTTP authentication

Mech supports performing a sequence of page fetches including following links and submitting forms. Each fetched page is parsed and its links and forms are extracted. A link or a form can be selected, form fields can be filled and the next page can be fetched. Mech also stores a history of the URLs you've visited, which can be queried and revisited.

See the home page for complete details.

366 questions
3
votes
2 answers

WWW::Mechanize->get giving error (HTTPS not supported)

I am trying to use WWW::Mechanize to login to Yahoo. However, when I try to get the login page, it gives the error Error GETing https://login.yahoo.com/config/login_verify2?&.src=ym: Protocol scheme 'https' is not supported (LWP::Protocol::https not…
Johnny
  • 1,419
  • 3
  • 13
  • 13
3
votes
2 answers

WWW::Mechanize SSL connect attempt failed for https get

I am trying to retrieve a Web page using WWW::Mechanize and it is failing with SSL connect error. How do I fix the issue? I am running ActivePerl 5.20.2 on Windows 10 x64. Here is the script I executed: perl -MIO::Socket::SSL=debug4…
scorpio
  • 379
  • 2
  • 11
3
votes
1 answer

Bad chunk-size in HTTP response: Net/HTTP/Methods.pm line 542

Questions that pose a similar problem: Issues with LWP when using HTTP/1.1: bad chunk-size, truncated responses. I am using the Perl module WWW::Mechanize to scrape web sites. As far as I understand, WWW::Mechanize uses the Net::HTTP module to…
Athithyaa
  • 33
  • 1
  • 6
3
votes
2 answers

How do I do a chunked transfer-encoding upload with WWW:Mechanize?

I'm attempting to use a particular web service, and I can successfully perform the upload with the following command: curl -X POST --header "Transfer-Encoding: chunked" -d @Downloads/file.pdf https://some.webservice/upload I get back a json…
John O
  • 4,863
  • 8
  • 45
  • 78
3
votes
2 answers

How to get WWW::Mechanize::Firefox working on OS X -- change ports?

I've installed the mozrepl add-on in Firefox for Mac OS X, but any attempt to use WWW::Mechanize::Firefox fails in new() with the message Failed to connect to , pattern match timed-out at /Library/Perl/5.10.0/MozRepl/Client.pm line 151 My…
user506491
  • 31
  • 2
3
votes
2 answers

Why does WWW::Mechanize GET certain pages but not others?

I'm new to Perl/HTML things. I'm trying to use $mech->get($url) to get something from a periodic table on http://en.wikipedia.org/wiki/Periodic_table but it kept returning error message like this: Error GETing …
Z.Zen
  • 838
  • 1
  • 10
  • 21
3
votes
2 answers

Visit Half Million Pages with Perl

Currently I'm using Mechanize and the get() method to get each site, and check with content() method each mainpage for something. I have a very fast computer + 10Mbit connection, and still, it took 9 hours to check 11K sites, which is not…
snoofkin
  • 8,725
  • 14
  • 49
  • 86
3
votes
1 answer

Session expires when using WWW::Mechanize on GeoServer

I try to use WWW::Mechanize in order to automate a session with GeoServer. GeoServer comes with a REST API, which can be used with curl. But at the moment, it is impossible to create a datastore for ImageMosaicJDBC with the REST API, so i would like…
MaxiReglisse
  • 3,093
  • 1
  • 13
  • 14
3
votes
2 answers

How do I find "wide characters" printed by perl?

A perl script that scrapes static html pages from a website and writes them to individual files appears to work, but also prints many instances of wide character in print at ./script.pl line n to console: one for each page scraped. However, a brief…
Larry Wang
  • 986
  • 6
  • 17
3
votes
2 answers

Saving a file with WWW::Selenium

Im trying to download a file with perl's WWW::Selenium. I get a popup box asking me if I want to save/open the file. I want to manipulate it and say 'save' at some given location. Im not sure how this can be done. Please help. P.S: I could not use…
user343409
  • 71
  • 3
3
votes
1 answer

Automatic Search Using WWW::Mechanize

I am trying to write a Perl script which will automatically key in search variables on this LexisNexis search page and retrieve the search results. I am using the WWW::Mechanize module but I am not sure how to figure out the field name of the search…
Amritha
  • 795
  • 3
  • 9
  • 26
3
votes
2 answers

WWW::Mechanize doesn't like POSTing json

I have the following code: my $j = encode_json { "arguments" => { "peer-port" => "4444" }, "method" => "session-set", }; $mech->get('http://192.168.1.10:9091'); my $req = HTTP::Request->new( 'POST', 'http://192.168.1.10:9091/transmission/rpc'…
John O
  • 4,863
  • 8
  • 45
  • 78
3
votes
3 answers

What is the preferred method of accessing WWW::Mechanize responses?

Are both of these versions OK or is one of them to prefer? #!/usr/bin/env perl use strict; use warnings; use WWW::Mechanize; my $mech = WWW::Mechanize->new(); my $content; # 1 $mech->get( 'http://www.kernel.org' ); $content = $mech->content; print…
sid_com
  • 24,137
  • 26
  • 96
  • 187
3
votes
1 answer

WWW::Mechanize Perl login only works after relaunch

I'm trying to login automatically in a website using Perl with WWW::Mechanize. What I do is: $bot = WWW::Mechanize->new(); $bot->cookie_jar( HTTP::Cookies->new( file => "cookies.txt", autosave => 1, …
Klaus
  • 1,241
  • 4
  • 14
  • 31
3
votes
2 answers

How can I extract all links from the page excluding one using Perl's WWW::Mechanize?

I'm trying to use WWW::Mechanize to extract some links from the HTML page using find_all_links() method. It supports matching on these criterias: text text_regex url url_regex url_abs url_abs_regex ... How can I extract all…
planetp
  • 14,248
  • 20
  • 86
  • 160
1 2 3
…
24 25