Stack Overflow
Stack Overflow

Questions tagged [mechanize-python]

Stateful programmatic web browsing in Python, after Andy Lester’s Perl module WWW::Mechanize.

Stateful programmatic web browsing in Python, after Andy Lester’s Perl module WWW::Mechanize.

Some of the key features:

  • The browser class mechanize.Browser implements the interface of urllib2.OpenerDirector, so any URL can be opened not just http.
  • Easy HTML form filling.
  • Convenient link parsing and following.
  • Browser history (.back() and .reload() methods).
  • The Referer HTTP header is added properly (optional).
  • Automatic observance of robots.txt (can be disabled if desired).
  • Automatic handling of HTTP-Equiv and Refresh.

python-mechanize was created by John J. Lee. Maintenance was taken over by Kovid Goyal.

HomePage | Documentation | Github | FAQ

235 questions
4
votes
1 answer

Web Scraper: Limit to Requests Per Minute/Hour on Single Domain?

I'm working with a librarian to re-structure his organization's digital photography archive. I've built a Python robot with Mechanize and BeautifulSoup to pull about 7000 poorly structured and mildy incorrect/incomplete documents from a collection. …
Cody Hess
  • 1,777
  • 15
  • 19
4
votes
1 answer

Programmatically Provide Secondary WiFi HotSpot Credentials with Python on Linux

I have credentials for a WiFi Service that is "Open" in the sense that in order to associate with it, a user is not required to provide a password. There is a secondary authentication method required to actually access the internet, and in order to…
PhilBot
  • 748
  • 18
  • 85
  • 173
4
votes
1 answer

can mechanize be fully replaced with requests [python]

I found a python2 project on GitHub, and wanted to transalte it into python3. I just wondered if requests and mechanize are basically the same, and can be replaced with each other.
Tsumugi Kotobuki
  • 103
  • 1
  • 1
  • 10
4
votes
2 answers

mechanize open Url python

I am trying to open a URL using mechanize in python. The code executes with no errors, but nothing actually happens. What am I missing? Also, is there a way to set the browser? This is python 2.7. import mechanize url='http://www.google.com/' op =…
user2242044
  • 8,803
  • 25
  • 97
  • 164
4
votes
1 answer

Find string in html page using mechanize

I am trying to find if a given string, lets say "Hello" exists in a given page. So far I have the following: agent = Mechanize.new page = agent.get('http://www.google.de/') and what should I do now? I have seen the search method, but it only…
Elchin
  • 584
  • 6
  • 19
3
votes
0 answers

Python Mechanize response is empty

I need the response in order to make Python decide what to do next, but the response keeps being empty. What could be the problem? import mechanize br =…
Jorrit Wegman
  • 93
  • 1
  • 7
3
votes
1 answer

python: module has no attribute mechanize

#!/usr/bin/env python import mechanize mech = mechanize.Browser() page = br.open(SchoolRank('KY')) Gives: Traceback (most recent call last): File "mechanize.py", line 2, in import mechanize File…
justin cress
  • 1,745
  • 5
  • 24
  • 35
3
votes
0 answers

Python / Mechanize read data from select box

I want to make a script that can automatically fill in a web form. Now there is 1 field that will not work. It's the only field (selectbox) that gets filled with JavaScript. Other selectboxes are working fine. This is my script: import…
Hoaxr
  • 73
  • 2
  • 7
3
votes
1 answer

Getting past a log-in screen with Python's mechanize

EDIT (10/30): solution found at the bottom of this post. Hello everyone, I'm new to the 'web-scraping' scene, and have been attempting to scrape data from pages at GISIS with Python. Though I originally attempted to do this with requests,…
Alex M
  • 63
  • 1
  • 7
3
votes
1 answer

Mechanizer in python - selecting form field with no name

I have a problem similar to Selecting an unnamed text field in a mechanize form (python) and Use mechanize to submit form without control name . I want to scrape the data of a website behind a login screen. However, I don't know how to select a form…
RNRug
  • 55
  • 5
3
votes
0 answers

Bypassing SSL certificate in Mechanize

I'm trying to Automate logins to the hardware web interface that we have on Site. The hardware has a url like this: https://192.168.100.207/login.shtm. This url has an invalid certificate and the browsers will warn me about a possible "Man in the…
khyal2008
  • 51
  • 6
3
votes
2 answers

Getting an error when importing mechanize branch for python3

I have installed mechanize library for python3. https://github.com/adevore/mechanize/tree/python3 But, when I import it, I get this error. Python 3.3.3 (default, Dec 30 2013, 16:15:14) [GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.2.79)] on…
crzyonez777
  • 1,789
  • 4
  • 18
  • 27
3
votes
2 answers

Method to automate Arduino Yun WiFi Login to Captive Portal with python or other?

What is the simplest path toward automating a WiFi login from an Arduino Yun (Yún) to a "Captive Portal" authenticated login method. I would like to stay in python, if that is the most appropriate. So for I've considered: curl wget (Stackoverflow…
Andrew Ward
  • 171
  • 2
  • 13
3
votes
1 answer

check errors or success in python mechanize

How do I check whether the submit has been successful or not. When I give correct username it does not show anything. But even if I submit wrong username, it does not throw any errors. How to catch the errors or check success results. Any help will…
Robin
  • 5,366
  • 17
  • 57
  • 87
3
votes
3 answers

Forcing Mechanize to use SSLv3

How would you force mechanize to use SSLv3 for HTTPS URLs that require it? If I try to use mechanize with all SSLv3-only URLs, I get the error: URLError:
Cerin
  • 60,957
  • 96
  • 316
  • 522
1
2
3
…
15 16