Stack Overflow
Stack Overflow

Questions tagged [www-mechanize]

WWW::Mechanize is a Perl module for programmatic web browsing, used for automating interaction with websites.

Features include:

  • All HTTP methods
  • High-level hyperlink and HTML form support, without having to parse HTML yourself
  • SSL support
  • Automatic cookies
  • Custom HTTP headers
  • Automatic handling of redirections
  • Proxies
  • HTTP authentication

Mech supports performing a sequence of page fetches including following links and submitting forms. Each fetched page is parsed and its links and forms are extracted. A link or a form can be selected, form fields can be filled and the next page can be fetched. Mech also stores a history of the URLs you've visited, which can be queried and revisited.

See the home page for complete details.

366 questions
0
votes
3 answers

Populate webpage textboxes using Perl script.How?

I've a webpage which has 2 text boxes. When i open the webpage cursor points to the first textbox. I need to fill these two textboxes and click on submit. I've written the code, but not working. I don't know the textboxes names as its on the server…
ramki067
  • 69
  • 1
  • 6
0
votes
1 answer

Forbidden 403 in perl script

I am trying to search and find a content from a site by using Perl Mechanize.It worked fine in the beginning after few execution i am getting 403 Forbidden instead of the search results, $m =…
Balakumar
  • 650
  • 1
  • 12
  • 29
0
votes
1 answer

How to use Perl to download a file which need a code confirm?

I just begin to learn some Perl based web application, however, I encounter an problem, I try to write a Perl script to download a file, which need input a code before download it. as an example, please see this url:…
Coeus Wang
  • 245
  • 2
  • 9
0
votes
1 answer

How can I follow a link that contains 2 things using Mechanize in Perl?

I want to follow a link that contains $foo AND $bar. I've tried this and it didn't work. $mech->follow_link( url_regex => qr/$foo/i && url_regex => qr/$bar/i)
Robert
  • 11
  • 1
  • 1
0
votes
1 answer

Perl web scraping

I am a Perl beginner and I am passionate about web scraping using Perl. After spending a couple of hours I wrote the code below for scraping company name, addresses and telephone number from yell.com. The script is working fine and I successfully to…
user1586957
  • 149
  • 2
  • 13
0
votes
2 answers

How to login and then submit another post with WWW::Mechanize in Perl

I have a PHP page (index.php) where, after the login with a form, I can send an SMS throughout another form. Through WWW::Mechanize I was able to login (if I print the response it correctly shows the second form to send the SMS) but I am not able to…
raz3r
  • 3,071
  • 8
  • 44
  • 66
0
votes
1 answer

Parsing links with Perl regex

I'm parsing a website with WWW::Mechanize to download some images. I need to populate an array with all links related to the resolutions available of the given image. But I need to populate only with links that are equal or less than '1440x900', but…
XVirtusX
  • 679
  • 3
  • 11
  • 30
0
votes
3 answers

Perl Click Button with WWW::Mechanize

I'm new to web scraping with Perl, and I'm trying to submit the page by filling in username and password fields, and clicking submit. I inspected the HTML code of the button in question, and it looks like:
aquemini
  • 950
  • 2
  • 13
  • 32
0
votes
2 answers

Perl Mechanize throwing connection error while using proxy

I am trying to download pdf files present in a website using perl MECHANIZE module. It worked fine when i am running the program without any proxy connection. But i am getting the below error when i tried to run the program on the company server…
Fla-Hyd
  • 279
  • 7
  • 17
0
votes
1 answer

Establish a session to call URL with Perl

I am trying to mine data from a webpage with the WWW::Mechanize perl module. However, I first need to establish a connection so that this webpage will allow me to access the data. In a browser, I can establish this connection by clicking a…
rodms
  • 371
  • 3
  • 12
0
votes
1 answer

how can i write text into a textbox using WWW::Mechanize?

i have been trying to write text into a textbox using WWW::Mechanize, i am not able to figure out a way to enter text into a textbox, does WWW::Mechanize support such feature,
Vidur Oberoi
  • 75
  • 1
  • 3
  • 12
0
votes
1 answer

Does set_visible in perl WWW:mechanize alter the page content

I am calling set_visible and the return value is non-zero. But when I print the content, it is the same as the default. Is this the intended value. Also, if I have javascript that will change another field, when will that execute (or does it)?
jamesatha
  • 7,280
  • 14
  • 37
  • 54
0
votes
1 answer

perl www mechanize and JSON

How to submit a valid JSON request using perl mechanize module I tried use WWW::Mechanize; use JSON; my $mech=WWW::Mechanize->new( stack_depth => 10, timeout => 120, autocheck => 0, ); $mech->agent_alias( 'Windows…
Shanthi
  • 637
  • 2
  • 7
  • 17
0
votes
1 answer

Selecting only from a specific table on a page WWW::Mechanize and CSS Selectors?

Good day, I am scraping a number of pages that display the data I require in tables. On the page there are multiple tables with the following: The items I want scraped are in table…
tread
  • 10,133
  • 17
  • 95
  • 170
0
votes
2 answers

Random error with WWW::Mechanize: Protocol scheme 'https' is not supported (LWP::Protocol::https not installed)

I'm working with WWW::Mechanize to slurp a products catalog from a web site into our database (Ingram Micro). Everything is over SSL. I'm receiving a random error like the following: Protocol scheme 'https' is not supported (LWP::Protocol::https not…
Francisco Zarabozo
  • 3,676
  • 2
  • 28
  • 54
1 2 3
…
24 25