5

I currently run browser tests via PhantomJS + Selenium in Python.

desired_capabilities = dict(DesiredCapabilities.PHANTOMJS)
desired_capabilities["phantomjs.page.settings.userAgent"] = ("Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36")
driver = webdriver.PhantomJS(executable_path="./phantomjs", desired_capabilities=desired_capabilities)
driver.get('http://google.com')

This works fine, unless the page I'm trying to get has a redirect on it.

Example:

https://login.vrealizeair.vmware.com/

In this case, the get doesn't work properly. The page source is empty: <html><head></head></body></html>.

This is a known issue with solutions posted that involve adding a snippet of code to handle redirects appropriately.

How/where do you add this code if you're running tests with Selenium (in my first code snippet)? Is it part of desired_capabilties?

Example:

page.onNavigationRequested = function(url, type, willNavigate, main) {
    if (main && url!=myurl) {
        myurl = url;
        console.log("redirect caught")
        page.close()
        renderPage(url);
    }
};

page.open(url, function(status) {
    if (status==="success") {
    console.log(myurl);
        console.log("success")
            page.render('yourscreenshot.png');
            phantom.exit(0);
    } else {
        console.log("failed")
            phantom.exit(1);
    }
});

I tried it with PhantomJS 1.9.8 and 2.0.1-development.

Artjom B.
  • 61,146
  • 24
  • 125
  • 222
okoboko
  • 4,332
  • 8
  • 40
  • 67

3 Answers3

5

It turns out the page couldn't be crawled due an error: SSL handshake failed.

The solution is to use the following line to initialize the driver:

driver = webdriver.PhantomJS(executable_path="./phantomjs", service_args=['--ignore-ssl-errors=true'])
okoboko
  • 4,332
  • 8
  • 40
  • 67
1

I have used the below settings:

DesiredCapabilities capabilities;
capabilities = new DesiredCapabilities();       
capabilities.setJavascriptEnabled(true);
capabilities.setCapability(PhantomJSDriverService.PHANTOMJS_EXECUTABLE_PATH_PROPERTY, "drivers/phantomjs.exe");
capabilities.setCapability(PhantomJSDriverService.PHANTOMJS_PAGE_SETTINGS_PREFIX,"Y");
capabilities.setCapability("phantomjs.page.settings.userAgent", "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:16.0) Gecko/20121026 Firefox/16.0");

//intialize driver and set capabilties

driver = new PhantomJSDriver(capabilities);

Then, I did the executed the following two lines and they worked fine for me

driver.get("https://login.vrealizeair.vmware.com/");
System.out.println(driver.getCurrentUrl());
System.out.println(driver.getPageSource());

Here's the output:

https://login.vrealizeair.vmware.com/sso/UI/Login
<!-- [RESPONSE_PAGE_TYPE=3DLOGIN] --><!DOCTYPE html><html><head>
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <title>Login | vRealize™ Air™</title>
    <link rel="stylesheet" href="/sso/css/styles.css?v=3" type="text/css">
    <link rel="shortcut icon" href="/sso/images/vmwareFavicon.ico" type="image/x-icon">

    <script async="" src="//rum-static.pingdom.net/prum.min.js"></script><script>...........................................
.....................................................
...................................................//Entire page source was displayed

I tried out the following code in python and it seems to be working fine:

from selenium import webdriver

driver = webdriver.PhantomJS("./phantomjs") 

driver.get("https://login.vrealizeair.vmware.com/")
print 'done'
print driver.current_url
print driver.page_source

Output (working fine):

https://login.vrealizeair.vmware.com/sso/UI/Login
<!-- [RESPONSE_PAGE_TYPE=3DLOGIN] --><!DOCTYPE html><html><head>
        <meta charset="utf-8">
        <meta http-equiv="X-UA-Compatible" content="IE=edge">
        <title>Login | vRealize™ Air™</title>
        <link rel="stylesheet" href="/sso/css/styles.css?v=3" type="text/css">

Imp note: Start navigating from the base page. The html code is empty because the website is probably throwing a 403 error. If the login URL is not working for you, try navigating from the pages that appear before the login page.

LittlePanda
  • 2,496
  • 1
  • 21
  • 33
  • I copied your exact code, but in Python. Basically, the user agent) and adding this `desired_capabilities["javascriptEnabled"] = True` - still no luck. Output: ``. Any thoughts? – okoboko Mar 31 '15 at 05:35
  • I have faced this same situation for one website when I was trying to directly navigate to page 2. Some websites wont allow your code to directly navigate to page n without navigating through the previous pages. You have to make sure that you code starts navigating from the base page - http://login.vrealizeair.vmware.com. This is something about the same origin policy. If navigating to the login page is giving you an empty html, try navigating to the page that occurs before the login page. – LittlePanda Mar 31 '15 at 07:10
  • I'm on 1.9.8. This page should work when accessed directly - no base page(s) required. – okoboko Mar 31 '15 at 14:14
  • I am using PhantomJS v2. Upgrade to v2 and it should work fine for you too. – LittlePanda Mar 31 '15 at 14:17
  • I tried the exact code you posted (for Python) using PhantonJS 2.0.1-development and still receive `` as the page source. Here's the contents of ghostdriver.log: https://gist.github.com/anonymous/5e5ea41a7704414514e5 – okoboko Apr 01 '15 at 04:10
  • I don't see anything wrong in the log. Have you tried navigating to any other url on this domain ? – LittlePanda Apr 01 '15 at 04:21
  • Yes, I have similar issues navigating to https://login.vrealizeair.vmware.com/sso/UI/Login. Interestingly, I can navigate to https://home.vrealizeair.vmware.com/cmpaas/public/forgotpassword/ just fine - same domain/server. So, there must be something on those two pages that's preventing it from working. – okoboko Apr 01 '15 at 04:23
  • Is it possible to navigate to the login page from the home page ? Or it might be a network issue maybe proxy – LittlePanda Apr 01 '15 at 04:26
1

This solution really worked for me, I was getting below error in the phantomjsdriver.log and on attempting to login, phantomjs was loggin out.

[DEBUG - 2017-08-19T20:37:59.288Z] Session [47739640-851e-11e7-9326-9bef0ad085f5] - page.onResourceError - {"errorCode":299,"errorString":"Error transferring https://int-test-cc.gcsip.nl:4443/rest/user/keepAlive?cacheBuster=1503175078533 - server replied: Unsupported Media Type","id":9,"status":415,"statusText":"Unsupported Media Type","url":"IPAdd:port/rest/user/keepAlive?cacheBuster=1503175078533"}

after adding below capabilities to phantomjs it worked -

caps.setJavascriptEnabled(true)
caps.setCapability(PhantomJSDriverService.PHANTOMJS_EXECUTABLE_PATH_PROPERTY, "phantomjs")
caps.setCapability(PhantomJSDriverService.PHANTOMJS_PAGE_SETTINGS_PREFIX,"Y");
caps.setCapability("phantomjs.page.settings.userAgent","Mozilla/5.0 (X11; Linux x86_64; rv:46.0) Gecko/20100101 Firefox/46.0")//"Mozilla/5.0 (Unknown; Linux x86_64) AppleWebKit/602.1 (KHTML, like Gecko) PhantomJS/2.5.0-development Version/9.0 Safari/602.1")
caps.setCapability(PhantomJSDriverService.PHANTOMJS_PAGE_CUSTOMHEADERS_PREFIX + "Content-Type","application/json;charset=utf-8")
caps.setCapability(PhantomJSDriverService.PHANTOMJS_PAGE_CUSTOMHEADERS_PREFIX + "Connection","Keep-Alive")