Stack Overflow
Stack Overflow

Questions tagged [mechanize-python]

Stateful programmatic web browsing in Python, after Andy Lester’s Perl module WWW::Mechanize.

Stateful programmatic web browsing in Python, after Andy Lester’s Perl module WWW::Mechanize.

Some of the key features:

  • The browser class mechanize.Browser implements the interface of urllib2.OpenerDirector, so any URL can be opened not just http.
  • Easy HTML form filling.
  • Convenient link parsing and following.
  • Browser history (.back() and .reload() methods).
  • The Referer HTTP header is added properly (optional).
  • Automatic observance of robots.txt (can be disabled if desired).
  • Automatic handling of HTTP-Equiv and Refresh.

python-mechanize was created by John J. Lee. Maintenance was taken over by Kovid Goyal.

HomePage | Documentation | Github | FAQ

235 questions
0
votes
2 answers

How to select radio button using multi-mechanize python

I have a simple php form and I would like to select the radio button using multimechanize, I'm able to select fill the text fields, but I'm not sure how to work with radio buttons or checkboxes... I tried the following but it doesn't work #…
msiles
  • 657
  • 3
  • 10
  • 19
0
votes
1 answer

Python browser.submit() not working

When i am doing browser.submit() ,its showing this error Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.7/dist-packages/mechanize-0.2.5-py2.7.egg/mechanize/_mechanize.py", line 541, in submit …
shailesh gupta
  • 427
  • 1
  • 5
  • 14
0
votes
1 answer

xpath returns empty in getting google search follow up links when using mechanize

br.open('http://www.google.com/advanced_search') br.select_form(name='f') br.form['as_q'] = "lxml" data = br.submit() html_string = data.read() //this is my input parser = etree.HTMLParser() tree = etree.parse(StringIO(html_string),…
user
  • 141
  • 1
  • 10
0
votes
1 answer

Python - Mechanize - import failure (python 3.3 on MacOS 10.7)

Installed mechanize successfully on MacOS 10.7 with Python3.3. But when I try it I get failure. >>> import mechanize Traceback (most recent call last): File "", line 1, in File…
Andrew Ward
  • 171
  • 2
  • 13
0
votes
0 answers

Python + Mechanize + Tor ( HTTPS ) = is end-to-end encrypted preventing eavesdroping?

I want to know if the following Python code is vulnerable to end-to-end eavesdropping attacks when the url points to a https:// site but this one does not support SSL encryption. Why I'm in doubt? because Mechanize internally uses urllib2 but HTTPS…
Fallouturama
  • 109
  • 1
  • 1
  • 9
0
votes
2 answers

How to set timeout with python-mechanize?

I'm using python-mechanize to scrape some web sites, which sometime simply don't respond to requests and these requests stay open too long, so I need to limit timeout for these requests. While using urlopen method, the timeout can be set using…
Michal Čihař
  • 9,799
  • 6
  • 49
  • 87
0
votes
3 answers

How to get images from Google's search results using Mechanize

I was trying to parse the HTML of Google's image search result and get the original link of the images. So far I was successful in writing a Python code to get the HTML of Google's search using Python's Mechanize and BeautifulSoup. Looking at…
Mercurial
  • 3,615
  • 5
  • 27
  • 52
0
votes
1 answer

Passing results from mechanize to BeautifulSoup

I get an when i try to mix mechanize and BeautifulSoup in the following code: from BeautifulSoup import BeautifulSoup import urllib2 import re import…
coverflower
  • 71
  • 1
  • 1
  • 5
0
votes
1 answer

Identify Browser alert messges in Mechanize - Python

I am trying to develop a small automated tool in python that can check Forms inputs of a web application for XSS vulnerability. I hope to do this using python mechanize library so that I can automate form filling and submit and get the response from…
Deepal
  • 1,729
  • 5
  • 24
  • 34
0
votes
1 answer

Best way to test if file available via URL?

I'm writing a script to DL the entire collection of BBC podcasts from various show hosts. My script uses BS4, Mechanize, and wget. I would like to know how I can test if a request for a URL yields a response code of '404' form the server. I have…
uncle-junky
  • 723
  • 1
  • 8
  • 33
0
votes
1 answer

Python Beautiful Soup parsing a UTF-8 coded table (using mechanize)

I'm trying to parse the following table, coded in UTF-8 (this is part of it):
erantdo
  • 685
  • 2
  • 9
  • 19
0
votes
1 answer

Import unresolved: Mechanize

I am working on a project that requires retrieving information from a webpage. Since the webpage uses javascript redirects, my friend recommended I use mechanize in Python. (Can anyone confirm that mechanize can handle javascript redirects or…
user2821275
0
votes
1 answer

mechanize: first form works, then "unknown GET form encoding type 'utf-8'"

I am trying to fill out 2 forms from the EUR-Lex website in order to record some data from the generated webpage. I am stuck at form #2. I get the feeling this should be easy and I've researched a bit, but no luck. import mechanize froot = '...' f…
mkln
  • 14,213
  • 4
  • 18
  • 22
0
votes
1 answer

Better way to auth to services

I am learning web-scraping with python-mechanize. At the moment, to enter a secure site, I have been entering data into forms manually then submitting. Like this: br.open("www.example.org/login.hmtl") br.select_form(nr=0) br['uname'] =…
pixelpax
  • 1,435
  • 13
  • 22
0
votes
1 answer

Is there a way to tell if a page opened with Mechanize isn't returning "search results"?

I am using Mechanize to log in to a web site and make a search. After extracting the links/info I want, I then recurisively move from the current page to the next to the next page. What I'm wondering is if there's an easy way to tell -- based on…
Bee Smears
  • 803
  • 3
  • 12
  • 22
1 2 3
…
15 16