Stack Overflow
Stack Overflow

Questions tagged [mechanize-python]

Stateful programmatic web browsing in Python, after Andy Lester’s Perl module WWW::Mechanize.

Stateful programmatic web browsing in Python, after Andy Lester’s Perl module WWW::Mechanize.

Some of the key features:

  • The browser class mechanize.Browser implements the interface of urllib2.OpenerDirector, so any URL can be opened not just http.
  • Easy HTML form filling.
  • Convenient link parsing and following.
  • Browser history (.back() and .reload() methods).
  • The Referer HTTP header is added properly (optional).
  • Automatic observance of robots.txt (can be disabled if desired).
  • Automatic handling of HTTP-Equiv and Refresh.

python-mechanize was created by John J. Lee. Maintenance was taken over by Kovid Goyal.

HomePage | Documentation | Github | FAQ

235 questions
2
votes
1 answer

How do I close a mechanize browser?

I'm running into an error in mechanize for python in which valid links that I try to open with it gives me the following error: urlopen error [Errno 8] nodename nor servname provided, or not known And it's not a problem of mechanize not being able…
Edasaur
  • 397
  • 1
  • 8
  • 19
2
votes
1 answer

How to find the current URL in python mechanize?

I see the answer to how to get the current URL in mechanize for Ruby (mechanize how to get current url) but how would one do so for the python mechanize? The documentation doesn't seem very...thorough...
Edasaur
  • 397
  • 1
  • 8
  • 19
2
votes
1 answer

Changing the link in python mechanize

I am trying to write a python script that will generate the rank-list of my batch. For this I simply need to change the roll-number parameter of the link using inspect element feature in web-browser. The link(relative) looks something…
nishantsingh
  • 4,537
  • 5
  • 25
  • 51
2
votes
2 answers

python mechanize forms() err

I'm using Python 2.7.6 and mechanize 0.2.5 and I want to log in to 'dining.ut.ac.ir' (I have the username and password)- but when I try to run the below script to get the forms list: import mechanize br =…
Arman Mazdaee
  • 25
  • 7
2
votes
0 answers

How can i get fully loaded html through python-mechanize?

Hi I'm using python mechanize to get datas from webpages. I'm trying to get imgurl from google image search webpage to download search result images. Here's my code I fill search form as 'dog' and submit. (search 'dog') import mechanize import…
p4vi4n
  • 21
  • 3
2
votes
1 answer

Follow First 5 Pages of Google Results Python Mechanize

I'm currently grabbing only the first page of google results for a query, but I want to grab the first 5 pages. gets a string like: https://encrypted.google.com/search?hl=en&q=site%3Asomedomain.com&start=0 the variable urls gets all 10 results for…
CodeTalk
  • 3,571
  • 16
  • 57
  • 92
2
votes
3 answers

Python Mechanize - Open URL with hash mark

I'm using Python Mechanize to open an URL with this format... https://www.simplewebsite.com?view=discussions#/?page=2 When the page opens...it gets this... https://www.simplewebsite.com?view=discussions Completely bypassing what's after the "#"…
Blag
  • 21
  • 1
  • 4
2
votes
0 answers

Calling js function using Browser from mechanize in python

I am working on a web-scraping project. I am using Browser from mechanize in python. I need to call a JavaScript function with two arguments. The arguments are known. __doPostBack(arg1,arg2) How do i call this function from Python?
RK Poddar
  • 2,551
  • 5
  • 17
  • 19
2
votes
1 answer

Mechanize submit a form that creates a dynamic flie that needs to be downloaded

So far I have mechanize code that does this: goes to a site logs in submits a form heres where i hit problems. What I need it to do is to write the response (a file) to a local file. I am pretty clueless as far as python interacting with the file…
zggz12
  • 125
  • 3
  • 13
2
votes
1 answer

How to NOT use a proxy with Python Mechanize

I am currently using Python + Mechanize for retrieving pages from a local server. As you can see the code uses "localhost" as a proxy. The proxy is an instance of the Fiddler2 debug proxy. This works exactly as expected. This indicates that my…
Udo Klein
  • 6,784
  • 1
  • 36
  • 61
2
votes
1 answer

Why does Mechanize(-Python) seem to overlook some hidden form fields but not others?

I'm working with a form that has several fields, some text, and several hidden. The problem is that when I look at the list of fields that my mechanize.Browser object "sees", some important hidden fields are missing, but not all. According to the…
BlueBomber
  • 282
  • 2
  • 11
2
votes
1 answer

Python & mechanize: How to scrape through pages in a row?

my problem is as follows: I'm trying to write a scraper that runs through the order process of an airline ticketing website. So I want to scrape a couple of pages that depend on the results of the pages before (I hope you get what I mean). I am so…
julianschnell
  • 193
  • 1
  • 2
  • 8
2
votes
0 answers

How do I handle popups using mechanize-python?

I am trying to write a program to access this site. When both the password and ID are correct, the page redirects to another URL. When the user ID is wrong, it will notify the user via a popup and and when the password is wrong, it will not say…
Display Name
  • 947
  • 2
  • 10
  • 18
2
votes
0 answers

How do I change the value of a HTML check box input using Python Mechanize?

I am having trouble modifying the value of a check box using the Python library Mechanize. The problem is that the form will not perform the desired action due to the fact that the value is modified by JavaScript, which Mechanize does not…
bringmesupport
  • 21
  • 2
2
votes
0 answers

Can't select form with python using mechanize

I'm trying to select a form with python, but it isn't working. The page i am working with is: http://www.safilonet.safilo.com/ def login(self): url = self._baseUrl + '/webapp/commerce/safilo/jsp/logon.jsp' self._br.open(url, timeout=30.0) …
user1682794
  • 21
  • 1
1 2 3
…
15 16