So I have this Python script ready that is able to pull certain information pertaining to my interest from a website. Nothing heavy, just politely pulling some content from the website.
However, the problem I am facing is that this website needs you to first log in using a Google, Yahoo, or Facebook account. So it goes like this: You visit the website, you try to 'search', it redirects you to login using one of the 3 methods. If I select Yahoo, it opens a new small window where I'm taken to the Yahoo login page. After I have logged in, this small window closes and the website is redirected to a "logged in" state. I can then make the search for the content.
So far I have this code:
#!/usr/bin/python
import re
import mechanize
br = mechanize.Browser()
br.set_handle_robots(False)
br.addheaders = [('User-agent', 'Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.6')]
br.set_handle_robots(False)
response = br.open("https://login.yahoo.com/config/login?.intl=us&.src=openid&.partner=&.pd=c%3DmZmAFpe.2e7WuWzcHD2ZPYQ-%26ockey%3Dwww.truecaller.com%26op%3D1.&occrumb=WJkaVqkbAY.&.done=https%3A%2F%2Fopen.login.yahoo.com%2Fopenid%2Fop%2Fstart%3Fz%3DQ4oGKMkwehm3_lS0mMkG_vmEWIdTDxMFLsN4yAu3CTi7kdZwRxb1m4XQ4T3uNWkbrzy2cTKNIkoVr6rQfYbaVpJInQ5xwjPJ9L.wUuUBDadBNY2QeTdAfZxFbkz97JGHrUccJFAr8EqOe46s7DrfIhfT2vz66w7Pu7C0uOLe2I9vUpoqnZrbNhGaJk.XSc0AvpE1lGi_YSZHFF1iNFOrXO69JNifMOgHYBHFnRaZAzfpZ8FgAM.ohZd0jv6BEMcBE__7Om2KZmbylufiOZPgvikDySub1fYOQBF6UlKDoIaxKwK8lUD8jiCtoGE4vJFqsZknSw--%26.scrumb%3D0")
#assert br.viewing_html()
print response.get_data()
br.select_form(nr=0)
br["username"] = 'name'
br["passwd"] = 'pass'
response = br.submit()
print response.get_data()
I don't know where to go from here. How do I follow the successful redirect to the other website in logged in state to make my queries?
