I intend to use twill to fill out a form on one page, hit the submit button, and then use BeautifulSoup to parse the resulting page. How can I feed BeautifulSoup the HTML page? I assume I have to read the current url, but I do not know how to actually return the url in order to do so. I have tried twill's TwillBrowser.get_url(), but it only returns None.
Asked
Active
Viewed 1,319 times
2 Answers
0
Finally figured this out!
If you import twill like so:
import twill.commands as com
then the url =
url = com.browser.get_url()

nmu
- 1,442
- 2
- 20
- 40
0
For any future sufferers, I have found better luck in using mechanize instead of twill as twill is an un-updated thin shell for mechanize. The solution is as follows:
import mechanize
url = "foo.com"
br = mechanize.Browser()
br.open(url)
br.select_form(name = "YOURFORMNAMEHERE") #make sure to leave the quotation marks
br["YOURINPUTFIELDNAMEHERE"] = ["YOURVALUEHERE"] #this must be in a list even if it is only one value
response = br.submit()
print response.geturl()

purpleladydragons
- 1,305
- 3
- 12
- 28