-1

I have used mechanize and successfully logged into a user login page. Now I want to navigate the site to a specific page in the submenus. When I try this by opening the URL of the specific page after logging in, another login page comes up which I do not have a username and password for. This log in page does not usually show up when I am navigating the site on a web browser. How can I do this?

        import mechanize
        import webbrowser
        import cookielib

        usern = '****'
        passw = '****'

        br = mechanize.Browser()
        cj = cookielib.LWPCookieJar()
        br.set_handle_robots(False)
        br.addheaders = [('User-agent', 'Mozilla/5.0 X11; U; Linux i686; en-US; rv:1.9.0.1) Gecko/2008071615 Fedora/3.0.1-1.fc9 Firefox/3.0.1')]

        r = br.open("https://myunihub-1.mdx.ac.uk/cas-web/login?service=https%3A%2F%2Fmyunihub.mdx.ac.uk%2Fc%2Fportal%2Flogin")
        br.select_form(nr=0)
        br.form['username'] = usern
        br.form['password'] = passw
        br.set_cookiejar(cj)
        br.submit


        url = "https://misis.mdx.ac.uk/mislve/bwskfshd.P_CrseSchd"
        webbrowser.open_new(url)
Kumar Patel
  • 83
  • 2
  • 9
  • 1
    Welcome to StackOverflow. To give you reasonable help with your coding problems, we need you to follow the help documentation for posting questions. Please read and follow http://stackoverflow.com/help/how-to-ask and http://stackoverflow.com/help/mcve. Among other things, we need you to post your minimal code and the full error it elicits. – Prune Oct 04 '15 at 16:43
  • what's the error message? – garg10may Oct 04 '15 at 17:41

1 Answers1

0

Try to use cookies and pretend to be actual browser. Some sites doesn't allow automated scripts/robots to crawl their sites. But you can always tell them no no I'm actual browser. 

import cookielib
cj = cookielib.LWPCookieJar()
br.set_cookiejar(cj)

And let's pretend we are not a robot and a actual browser. 

br.set_handle_robots(False)
br.addheaders = [('User-agent', 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.1) Gecko/2008071615 Fedora/3.0.1-1.fc9 Firefox/3.0.1')]
garg10may
  • 5,794
  • 11
  • 50
  • 91
  • It still hasn't worked I added my code to the question – Kumar Patel Oct 04 '15 at 17:36
  • You can include the code in your question. Press CTRL + K on lines which are code that will format it. In case answer helps you can accept it and upvote. Also try to include the errors in the question. – garg10may Oct 04 '15 at 17:39
  • @KumarPatel you haven't included `br.set_cookiejar(cj)` this will set your cookie. – garg10may Oct 04 '15 at 17:41
  • br.set_cookiejar(cj) is already there just above br.submit() There is not error. It's just that when it opens up the page in the web browser, it asks me to log in again when I have already logged-in in the code. – Kumar Patel Oct 04 '15 at 17:47
  • Try to set it just above add.headers, before opening up the webpage. before br.open – garg10may Oct 04 '15 at 17:51
  • this hasn't made a difference :/ – Kumar Patel Oct 04 '15 at 17:57
  • The code works fine, there is no error. The problem is when the url opens in my web browser it asks me to log in. However I have already logged in previously and have accessed the portal. I want to access a submenu after logging in. When I try to do this another login page shows up which is not usually there when using a web browser. – Kumar Patel Oct 04 '15 at 18:05
  • Not sure, try to log out from your browser session, clean cookies etc., may be it's doesn't allow multiple sessions. Then try it through script. – garg10may Oct 04 '15 at 18:13