Stack Overflow
Stack Overflow

Questions tagged [www-mechanize]

WWW::Mechanize is a Perl module for programmatic web browsing, used for automating interaction with websites.

Features include:

  • All HTTP methods
  • High-level hyperlink and HTML form support, without having to parse HTML yourself
  • SSL support
  • Automatic cookies
  • Custom HTTP headers
  • Automatic handling of redirections
  • Proxies
  • HTTP authentication

Mech supports performing a sequence of page fetches including following links and submitting forms. Each fetched page is parsed and its links and forms are extracted. A link or a form can be selected, form fields can be filled and the next page can be fetched. Mech also stores a history of the URLs you've visited, which can be queried and revisited.

See the home page for complete details.

366 questions
0
votes
1 answer

How do I press a button with no name attribute with WWW::Mechanize?

This is the button I want to click: As you see it has no name at all to specify the order on it. I am using the WWW::Mechanize module. I tried $agent->submit();, but it seems to not do the job, any help…
Hady Saker
  • 21
  • 1
0
votes
1 answer

Perl Use of uninitialized value in addition (+) at.. WWW::Mechanize::Timed

I have the following very simple piece of code: #!/usr/bin/perl use strict; use warnings; use WWW::Mechanize::Timed; my $ua = WWW::Mechanize::Timed->new(); my $url = 'www.stackoverflow.com'; $ua->get($url); print "Total time: " .…
yonetpkbji
  • 1,019
  • 2
  • 21
  • 35
0
votes
2 answers

Extract several rows with with HTML::TableExtract

I have made one script which will extract all the Row data from HTML tags. I am having 30 HTML tags on my HTML page. Based on count, my code will fetch particular row data. Let's say if I need data present in 5th ..., then my…
user2106358
  • 21
  • 1
  • 2
0
votes
2 answers

Perl WWW::Mechanize Storing urls in a hash unless it has already been found

I have a question I'm hoping you could help with? foreach my $url ( keys %{$newURLs} ) { # first get the base URL and save its content length $mech->get($url); my $content_length = $mech->response->header('Content-Length'); # now iterate…
yonetpkbji
  • 1,019
  • 2
  • 21
  • 35
0
votes
1 answer

How can I select a random listbox item with WWW::Mechanize?

With Perl's WWW::Mechanize module, I want to select a random value from a select box. How can I do this? With dump_forms I can dump select box values, but how can I get them in an array?
Jazz
  • 5,747
  • 5
  • 43
  • 55
0
votes
3 answers

find_all_links() Perl function doesn't find all links

I'm starting with Perl and I'm trying to do a script that logs me out of a page (I already did the login one). The idea is, I'm trying to use find_link function from WWW::Mechanize (I also tried WWW::Mechanize::Firefox) to find the logout link, but…
Malincy Montoya
  • 87
  • 1
  • 13
0
votes
2 answers

How do I configure WWW::Mechanize to work through a proxy?

I sit behind a proxy and have HTTP and WWW::Mechanize working OK locally. But advice || pointers as to how to get through a proxy please. Regards Ian
Ian Pellew
0
votes
1 answer

www::mechanize perl, executable

I read a question you asked about 5months ago on stackoverflow.com more precisly here it is: Using WWW::Mechanize to navigate forms on Amazon site I am creating a script that enters a site and enters my credentials, to finally save the sites source…
user1757842
  • 9
0
votes
1 answer

Is there a python package which does what WWW::Mechanize does in perl?

Is there a python package which can handle the same tasks covered by WWW::Mechanize in perl?
APE
  • 73
  • 5
0
votes
1 answer

injecting HTTP::Response objects in a WWW::Mechanize instance

what is the proper way of injecting an HTTP::Response (and implicitly or maybe explicitly as well the corresponding HTTP::Request) in a WWW::Mechanize instance, if any? Basically I want to separate the downloading process so I can use…
user237419
  • 8,829
  • 4
  • 31
  • 38
0
votes
2 answers

Downloading several files froma website with Perl

I began to mess around with the Web in Perl. I use Windows, and ActivePerl. I wrote a script, which downloads mp3 files (It's an archive of a radio talkshow, all legal, in case you're wondering :) ) It (in theory) parses a website collects all…
librarian
  • 129
  • 10
0
votes
1 answer

Accessing hidden field values using Ruby Mechanize

I am trying to access the form values of hidden fields so I can set them or retrieve them. I tried the form.field_with() method as well as just form.fieldname and cannot figure out how to get a reference back for a hidden field.
xMythicx
  • 827
  • 1
  • 11
  • 27
0
votes
1 answer

Trouble with mechanize and submitting a text area

So I'm trying to submit a simple form that has one text area that looks similar to this "Lots of stuff that doesn't matter a couple of lines later agent => A5656,,5665" What I'm trying to do is take out the line with the agent. By the way, there…
David Hahn
  • 740
  • 9
  • 26
0
votes
1 answer

WWW::Mechanize in Perl, Script Gets Killed

I have written a Perl Script which uses WWW::Mechanize to connect to a site, login and then visit a few pages inside the site. It all works good, however, when I try to visit a large number of pages, the script gets killed. I am sure this has got…
Neon Flash
  • 3,113
  • 12
  • 58
  • 96
0
votes
1 answer

Downloading Text from Several Links using WWW::Mechanize

For an entire week I have been attempting to write a code that will download links from a webpage and then loop through each link to dump the content written on each link's page. The original webpage I downloaded has 500 links to separate web pages…
Miriam Herm
  • 62
  • 1
  • 7
1 2 3
…
24 25