Stack Overflow
Stack Overflow

Questions tagged [mechanize]

Mechanize is a library for automated web browsing originally developed for Perl, there are now also Python and Ruby implementations.

Mechanize is a Ruby library for automating interaction with websites. Mechanize automatically stores and sends cookies, follows redirects, can follow links, and submit forms. Form fields can be populated and submitted. Mechanize also keeps track of the sites that you have visited as a history. It is adapted from the Perl module. There is also a for Python.

2512 questions
13
votes
1 answer

How to let Ruby Mechanize get a page which lives in a string

Generally Mechanize will get a webpage from a URL and the result of the get method is a Mechanize::Page object, from which you can use a lot of useful methods. If the page lives in a string, how do I get the same Mechanize::Page object? require…
Just a learner
  • 26,690
  • 50
  • 155
  • 234
12
votes
1 answer

Cannot press button

I'm trying to code a bot for a game, and need some help to do it. Being a complete noob, I googled how to do it with python and started reading a bit about mechanize.
Luis Silva
  • 159
  • 4
12
votes
2 answers

Getting and trapping HTTP response using Mechanize in Python

I am trying to get the response codes from Mechanize in python. While I am able to get a 200 status code anything else isn't returned (404 throws and exception and 30x is ignored). Is there a way to get the original status code? Thanks
user319723
  • 451
  • 1
  • 5
  • 8
12
votes
3 answers

Silence loggers and printing to screen - Python

I'm having a problem with my python script. It's printing massive amounts of data on the screen, and I would like to prevent all sorts of printing to screen. Edit: The library I'm using is mechanize, and it's printing a LOT of data on screen. I…
RadiantHex
  • 24,907
  • 47
  • 148
  • 244
12
votes
5 answers

Scraping Data from Facebook with Python

I've been trying for several day now (unsuccessfully) to scrape cities from about 500 Facebook URLs. However, Facebook handles its data in a very strange way and I can't figure out what's going on under the hood to understand what I need to…
cscanlin
  • 178
  • 2
  • 8
  • 21
12
votes
2 answers

mechanize python click a button

I have a form with button and would like to be able to click it. I have tried mech.form.click("submit") but that gives the following error: ControlNotFoundError: no control matching kind 'clickable', id…
nunos
  • 20,479
  • 50
  • 119
  • 154
12
votes
1 answer

Downloading file with Python mechanize

I am trying to download a file from a website using python and mechanize. My current code successfully logs on to the website and opens the page that contains the download link. The download link is:…
user1137778
  • 1,111
  • 3
  • 14
  • 26
11
votes
3 answers

How can I keep WWW::Mechanize from following redirects?

I have a Perl script that uses WWW::Mechanize to read from a file and perform some automated tasks on a website. However, the website uses a 302 redirect after every time I request a certain page. I don't want to be redirected (the page that it…
rfusca
  • 7,435
  • 2
  • 30
  • 34
11
votes
2 answers

How to make mechanize wait for web-page 'full' load?

I want to scrape some web page which loads its components dynamically. This page has an onload script, and I can see the complete page 3-5 seconds after typing the URL into my browser. The problem is, when I call br.open('URL'), the response is the…
planetes853
  • 309
  • 1
  • 4
  • 11
11
votes
2 answers

How do you select choices in a form using Python?

I'd like to know how to select options in a form that is formatted like