Stack Overflow
Stack Overflow

Questions tagged [mechanize-python]

Stateful programmatic web browsing in Python, after Andy Lester’s Perl module WWW::Mechanize.

Stateful programmatic web browsing in Python, after Andy Lester’s Perl module WWW::Mechanize.

Some of the key features:

  • The browser class mechanize.Browser implements the interface of urllib2.OpenerDirector, so any URL can be opened not just http.
  • Easy HTML form filling.
  • Convenient link parsing and following.
  • Browser history (.back() and .reload() methods).
  • The Referer HTTP header is added properly (optional).
  • Automatic observance of robots.txt (can be disabled if desired).
  • Automatic handling of HTTP-Equiv and Refresh.

python-mechanize was created by John J. Lee. Maintenance was taken over by Kovid Goyal.

HomePage | Documentation | Github | FAQ

235 questions
0
votes
1 answer

Some nudges for a first-time scraper

I'm trying to programatically (in Python) retrieve account information from this website for a list of properties I have (identified by BRT number). This should be very simple, and I've read a few things I've found via Google, but it's all way over…
MichaelChirico
  • 33,841
  • 14
  • 113
  • 198
0
votes
1 answer

Python mechanize - submit form response

I have this code: import mechanize def register(): br = mechanize.Browser() br.set_handle_robots(False) br.open("http://somtehing.com/form2/index.php") br.select_form(nr=0) br.form["email"]…
Yerdno
  • 21
  • 1
  • 3
0
votes
1 answer

Use mechanize with python

I'm trying to open url with mechanize but not just open it and close it right away I want it to open the url then wait 7 minute then close the url. what I'm trying to do : import mechanize import cookielib import time url='http://google.com/' op =…
deounix
  • 141
  • 2
  • 4
  • 18
0
votes
1 answer

Best scraper/crawler needed for multipage forms (Nokogiri, Scrapy, other?)

I've read that Nokogiri/Mechanize (Ruby) for example are not good at traversing multiple pages, but may be better with sites that use Ajax. The sites I want to scrape are multi-page forms, with some ajax overlays. Speed is important. These sites…
hippypink
  • 17
  • 2
0
votes
2 answers

login.live.com with python and mechanize?

I need to automatically login with python and mechanize on login.live.com. The problem is,that I can't find any browser.forms(), but there should be some, since I checked the HTML code: My code: import urllib2 import lxml from mechanize import…
Nero Kocher
  • 1
  • 1
0
votes
1 answer

how do you access rows in tables in html form usibg python mechanize

im trying to access the rows in an html form, the html code in the form is as such
Adhil
  • 1,678
  • 3
  • 20
  • 31
0
votes
0 answers

Using mechanize in Python to log into Yahoo, then following a re-direct to another site

So I have this Python script ready that is able to pull certain information pertaining to my interest from a website. Nothing heavy, just politely pulling some content from the website. However, the problem I am facing is that this website needs you…
learnerX
  • 1,022
  • 1
  • 18
  • 44
0
votes
1 answer

Python Mechanize select value from