Stack Overflow
Stack Overflow

Questions tagged [mechanize-python]

Stateful programmatic web browsing in Python, after Andy Lester’s Perl module WWW::Mechanize.

Stateful programmatic web browsing in Python, after Andy Lester’s Perl module WWW::Mechanize.

Some of the key features:

  • The browser class mechanize.Browser implements the interface of urllib2.OpenerDirector, so any URL can be opened not just http.
  • Easy HTML form filling.
  • Convenient link parsing and following.
  • Browser history (.back() and .reload() methods).
  • The Referer HTTP header is added properly (optional).
  • Automatic observance of robots.txt (can be disabled if desired).
  • Automatic handling of HTTP-Equiv and Refresh.

python-mechanize was created by John J. Lee. Maintenance was taken over by Kovid Goyal.

HomePage | Documentation | Github | FAQ

235 questions
2
votes
1 answer

Python Mechanize.Browser.Open(url) Status Code

I recently moved from perl to python and am updating some of my scripts which used perls WWW::Mechanize to use python mechanize module. All good so far but I cant get find the HTTP Status code anywhere? (200, 301, etc) I've googled and found this…
James
  • 241
  • 3
  • 13
2
votes
1 answer

Verifying br.submit() using Python's Mechanize module

Just trying to login to a website using mechanize. When I print "br.form", I can see my credentials entered into my form. But I do not know how to actually submit the form properly. I use "br.submit()" and attempt to verify it has proceeded to the…
Mitch
  • 23
  • 1
  • 5
2
votes
1 answer

mechanize streaming download python

I am trying to download files from a site that requires htaccess - style authentication as well as a cookie, and am using python mechanize to download these files. is there a way to start writing before loading the whole file into memory? edit: I'm…
user1036542
  • 149
  • 1
  • 5
2
votes
3 answers

python - mechanize (setting input into form)

I found out how to retreive the html page of a topic from google search using a tutorial.This was given in the tutorial. import mechanize br = mechanize.Browser() br.open('http://www.google.co.in') br.select_form(nr = 0) I understood till this that…
Manoj
  • 961
  • 4
  • 11
  • 37
2
votes
1 answer

Python mechanize javascript

I'm trying to use mechanize to grab prices for New York's metro-north railroad from this site: http://as0.mta.info/mnr/fares/choosestation.cfm The problem is that when you select the first option, the site uses javascript to populate your list of…
jmetz
  • 815
  • 1
  • 9
  • 19
1
vote
2 answers

python mechanize session not saved

I'm trying to use python mechanize to retrive the list of apps on iTunes connect. Once this list is retrieved, further work will be done with those links. Logging in succeeds but then when i follow the "Manage Your Applications" link I get…
Inn0vative1
  • 2,014
  • 3
  • 27
  • 43
1
vote
1 answer

Python mechanize hangs on br.open(url) with self._sleep(pause) traceback

I have written a python function to rate websites according to some parameters (a series of words). The function uses Python Mechanize and it works fine most of the time. However, for some websites it just hangs there until I ctrl+c on the…
dtrujillo
  • 553
  • 6
  • 10
1
vote
1 answer

Overriding Python Mechanize.Browser.open() method

The following code: #!/usr/bin/env python import mechanize class MechanizeSubclass(mechanize.Browser): def…
Todd Hayton
  • 415
  • 1
  • 3
  • 9
1
vote
2 answers

Can't print a specific line from text file

So I currently have this code to read an accounts.txt file that looks like this: username1:password1 username2:password2 username3:password3 I then have this (thanks to a member here) read the accounts.txt file and split it at the username and…
Shtoops
  • 73
  • 1
  • 1
  • 4
1
vote
1 answer

Fill out Ajax forms in Python

I'm trying to fill forms of an ajax box (just my term for those several forms) using the mechanize module, but it seems not to work. I'm not a web programmer but afaik the ajax box updates itself 'onchange' with an event which is handled by the…
naeg
  • 3,944
  • 3
  • 24
  • 29
1
vote
0 answers

How to fill a form input field using its ID rather than the form name with mechanize?

I am trying to automate geocoding with python. There is a simple website that does this, but I don't want my user to manually visit the website. I want to automatically take his adress and convert it to latitude longitude using the website. I'd like…
WhiteWood
  • 103
  • 1
  • 6
1
vote
0 answers

Webscraping in Python a page with login and redirection

I am trying to login to a financial service I am a customer to retrieve automatically some data by using Python requests. I have been inspired by this page: import requests from typing import Dict def get_payload(username:str, password:str) ->…
Jean-Francois T.
  • 11,549
  • 7
  • 68
  • 107
1
vote
1 answer

OAuth2 Grant Type - authorization code (python)

I am try to figure out how to get Oauth 2 working in my python code. import requests, json import webbrowser authorize_url = "https://tcfhirsandbox.com.au/oauth2/authorize" token_url = "https://tcfhirsandbox.com.au/oauth2/token" state =…
Rusty
  • 11
  • 5
1
vote
0 answers

Can't find form name with python-mechanize

I just started playing around with python. Recently I stumbled upon web scraping and I thought that it would not be that hard to make a Kahoot (https://kahoot.it/) bot. You may or may not know what I'm talking about. That's not important now. What I…
jan havel
  • 11
  • 3
1
vote
1 answer

Transfering login from mechanize to urllib or requests

I'm currently trying to transfer some of my code to either urllib3 or requests. urllib3 would be the preferred method to reduce the modules imported. This is the code currently working. payload = ["", "" browser =…
shanks3042
  • 21
  • 3
1 2 3
…
15 16