Stack Overflow
Stack Overflow

Questions tagged [www-mechanize]

WWW::Mechanize is a Perl module for programmatic web browsing, used for automating interaction with websites.

Features include:

  • All HTTP methods
  • High-level hyperlink and HTML form support, without having to parse HTML yourself
  • SSL support
  • Automatic cookies
  • Custom HTTP headers
  • Automatic handling of redirections
  • Proxies
  • HTTP authentication

Mech supports performing a sequence of page fetches including following links and submitting forms. Each fetched page is parsed and its links and forms are extracted. A link or a form can be selected, form fields can be filled and the next page can be fetched. Mech also stores a history of the URLs you've visited, which can be queried and revisited.

See the home page for complete details.

366 questions
3
votes
3 answers

How can I make WWW:Mechanize to not fetch pages twice?

I have a web scraping application, written in OO Perl. There's single WWW::Mechanize object used in the app. How can I make it to not fetch the same URL twice, i.e. make the second get() with the same URL no-op: my $mech =…
planetp
  • 14,248
  • 20
  • 86
  • 160
3
votes
1 answer

Saving PDF files with WWW::Mechanize corrupts them

I'm trying to write a script that will log into Bank of America and download PDF statements. I've manage all the difficult tricks, and I'm hung up on saving the PDF files. I've tried both the ':content_file' => "some file path" method, and a…
John O
  • 4,863
  • 8
  • 45
  • 78
3
votes
2 answers

How can I get the frame source with Perl's WWW::Mechanize?

Using WWW::Mechanize::Firefox, I can get the source of the page I visited. However, if the page contains a frame, I get the frame tag and not the actual source of the page that is loaded. Mechanize::Frames seems to be what I am looking for. Is there…
SJaguar13
  • 131
  • 6
3
votes
2 answers

WWW::Mechanize : How to upload file under different file name?

How to make WWW::Mechanize upload file under different file name? I would like web server to see/record file name different from file name on my computer.
AnFi
  • 10,493
  • 3
  • 23
  • 47
3
votes
1 answer

perl WWW::Mechanize submit_form() write directly to a file?

I have a perl script that navigates through two form pages on a website in order to download the result of final form submission to a file. This page is very large and I'd rather write it directly to disk instead of having the whole thing sitting…
Jessa
  • 1,549
  • 2
  • 12
  • 26
3
votes
6 answers

Perl - Check if any elements in each different array matches a variable

I have a problem I am hoping someone can help with (greatly simplified for the purposes of explaining what I am trying to do)... I have three different arrays: my @array1 = ("DOG","CAT","HAMSTER"); my @array2 = ("DONKEY","FOX","PIG", "HORSE"); my…
yonetpkbji
  • 1,019
  • 2
  • 21
  • 35
3
votes
1 answer

Hosts file and Perl Mechanize

I am trying to make a perl script to send a get request to applications I have created. My applications are running on separate servers, so in order to make it seem like a domain, I added the ip's to my /etc/hosts file. My hosts file looks…
Eumcoz
  • 2,388
  • 1
  • 21
  • 44
3
votes
1 answer

WWW::Mechanize::Firefox troubleshooting

I have MozRepl installed in my browser and set to start on startup and accept outside connections. So my questions are as follows 1) will simply running the code my $mech = WWW::Mechanize::Firefox->new(); launch the firefox browser? 2) I have set…
Marcus Lim
  • 567
  • 2
  • 5
  • 14
3
votes
1 answer

WWW::Mechanize::Timed https timeout does not work

So I've researched to the ends of the internet (at least I think so) about this issue. I'm trying to set an alarm timeout of 60 seconds for a get() but it does not get caught and will run past 60 seconds, also any time the default timeout (180 sec)…
Jude_Quintana
  • 61
  • 7
3
votes
4 answers

Perl LWP::UserAgent hangs for 120 seconds seemingly randomly against a given server

I noticed a delay of approximately 120 seconds between handling the response_data and response_done events in WWW::Mechanize with a given https web site. I checked with a normal web browser and do not experience this slowness so I suspect there is…
Benoit
  • 76,634
  • 23
  • 210
  • 236
3
votes
3 answers

Perl Using WWW::Mechanize to set a radio button that has no value

I have to say I am new to Perl, maybe made about 3 or 4 scripts so far, and I find Perl decently easy to understand and use. Anyways, I have a problem with my current script. I have been stuck on this problem for a while and just can't seem to find…
imakeitrayne
  • 155
  • 1
  • 2
  • 11
3
votes
4 answers

How to check a checkbox using WWW::Mechanize?

I read the Perl WWW::Mechanize module and this is the syntax: $mech->tick( $name, $value [, $set] ) But, when I checked the page source of the web page, this is what I found:
GLOBAL_PROCESSING
…
cppcoder
  • 22,227
  • 6
  • 56
  • 81
2
votes
2 answers

WWW::Mechanize ignores base href on gzipped content

As the title says WWW::Mechanize does not recognize if page content iz gzipped. Here is an example: use strict; use warnings; use WWW::Mechanize; my $url = 'http://objectmix.com/perl/356181-help-lwp-log-after-redirect.html'; my…
toktok
  • 279
  • 1
  • 5
  • 13
2
votes
3 answers

Login to website with redirection using WWW::Mechanize

I know it sounds like a question which was already asked on forum ,however I believe that I have a slightly different scenario here.. I try to use Perl to log on to a web site which requires authentication. I already realized that the most…
Eugene S
  • 6,709
  • 8
  • 57
  • 91
2
votes
5 answers

Website button click - Perl WWW::Mechanize

I try to use the perl script to automate the interaction with a website. I use module WWW::Mechanize to realize my design. But I cannot perform the button click in my perl script by using command as below. $mech->click( $button [, $x, $y] )…
Tommy Liu
  • 239
  • 4
  • 13
1 2 3
…
24 25