2

I intend to use twill to fill out a form on one page, hit the submit button, and then use BeautifulSoup to parse the resulting page. How can I feed BeautifulSoup the HTML page? I assume I have to read the current url, but I do not know how to actually return the url in order to do so. I have tried twill's TwillBrowser.get_url(), but it only returns None.

purpleladydragons
  • 1,305
  • 3
  • 12
  • 28

2 Answers2

0

Finally figured this out!

If you import twill like so:

import twill.commands as com

then the url =

url = com.browser.get_url()

Source: http://nullege.com/codes/search/twill.commands.browser.get_url?utm_expid=24446124-0.lSQi4Ea5S7WZwxHvFPbOIA.0&utm_referrer=https%3A%2F%2Fwww.google.com%2F

nmu
  • 1,442
  • 2
  • 20
  • 40
0

For any future sufferers, I have found better luck in using mechanize instead of twill as twill is an un-updated thin shell for mechanize. The solution is as follows:

import mechanize

url = "foo.com"
br = mechanize.Browser()

br.open(url)

br.select_form(name = "YOURFORMNAMEHERE") #make sure to leave the quotation marks
br["YOURINPUTFIELDNAMEHERE"] = ["YOURVALUEHERE"] #this must be in a list even if it is only one value

response = br.submit()

print response.geturl()
purpleladydragons
  • 1,305
  • 3
  • 12
  • 28