Using mechanize to input username and password

Question

How do I use mechanize to input username and password onto this site?

I deleted and changed my post because my previous one had too much extra information

I've read in other posts that maybe this has to do with javascript, but how do I tell? and what do I do with that information?

import mechanize
import cookielib

url = 'https://www.pin1.harvard.edu/cas/login?service=https%3A%2F%2Fwww.pin1.harvard.edu%2Fpin%2Fauthenticate%3F__authen_application%3DFAS_AC_AUTHENTICATOR'
#req = requests.get(url)
#dom = web.Element(req.text)

#Handles all the browser details 
br = mechanize.Browser()
br.addheaders = [('User-agent', 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.1) Gecko/2008071615 Fedora/3.0.1-1.fc9 Firefox/3.0.1')]
#self.browser = mechanize.Browser(factory=mechanize.RobustFactory())

#Cookie Jar
cj = cookielib.LWPCookieJar()
br.set_cookiejar(cj)

# Browser options
br.set_handle_equiv(True)
br.set_handle_gzip(True)
br.set_handle_redirect(True)
br.set_handle_referer(True)
br.set_handle_robots(False)

br.open(url)


#Select First Form
#br.select_form(nr=1)

#br['username'] = '40839852'

#print list(br.forms())[0] 

for form in br.forms():
    print "Form name:", form.name
    print form
    break

br.select_form(name= formname)
br[searchname] = term
res = br.submit()
content = res.read()
dom = web.Element(content)

TRACEBACK

ParseError: unexpected '/' char in declaration
---> 32 for form in br.forms():
     33     print "Form name:", form.name
     34     print form

UPDATE - BASED ON PACO'S SUGGESTION I ADDED...But i still get a traceback. Python unable to retrieve form with urllib or mechanize

beg =  re.search(t, res.read()).span()[1]
res.set_data(res.get_data()[beg:])
br.set_response(response)
br.select_form(nr=0)


<ipython-input-25-bd1b73406b45> in <module>()
     28 br.set_response(response)
     29 
---> 30 br.select_form(nr=0)
     31 
     32 
ParseError: unexpected '-' char in declaration

long shot, but I'm guessing that `br.forms` isn't a function. What happens if you remove the parentheses? — Paul H, Apr 18 '14 at 23:26
Unfortunately, it is a function i think. http://wwwsearch.sourceforge.net/mechanize/ if you remove the parens, you get: > — user3314418, Apr 18 '14 at 23:27

taco · Accepted Answer · 2014-04-19T00:05:30.083

1

This is how I selected the first form in my code.

br.select_form(nr=0)
#Form fields to populate
br.form['username'] = username
br.form['password'] = password
#Submit the login form
br.submit()

Modify it to suit your needs. The "nr=0" is probably what you're looking for.

But the problem is the DOCTYPE. I tested the following, and it strips it out.

html = br.response().get_data().replace('<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd >', '')
response = mechanize.make_response(
    html, [("Content-Type", "text/html")],
url, 200, "OK")
br.set_response(response)

I took this straight from the Mechanize FAQ.

edited Apr 19 '14 at 00:05

answered Apr 18 '14 at 23:33

taco

1,367
17
32

unfortunately I still get the parseerror traceback saying unexpected '/' char in declaration. It's something to do with the select_form function --------------------------------------------------------------------------- ParseError Traceback (most recent call last) in () 23 24 ---> 25 br.select_form(nr=0) – user3314418 Apr 18 '14 at 23:34
What do I do if I do find that there are errors? (1) Mismatch between Public and System identifiers in the DOCTYPE declaration (2) No Character encoding declared at document level – user3314418 Apr 18 '14 at 23:38
by strip out, you mean to just delete it? – user3314418 Apr 18 '14 at 23:39
how did you know that ' – user3314418 Apr 19 '14 at 00:46
1

http://validator.w3.org/ will explain why when you put that URL into their validator - "Mismatch between Public and System identifiers in the DOCTYPE declaration", which can lead to a parsing error. Web browsers are more forgiving than Mechanize and other parsers. – taco Apr 19 '14 at 01:45

Using mechanize to input username and password

1 Answers1