3

Good day, I am working on school project to iterate through all users data in LinkedIn using PhantomJS.exe(phantomjs-2.1.1) on windows with JAVA (IntelijiIdea), with selenium (selenium-server-standalone-3.7.1.jar driver).

I found out that in LinkedIn there are many elements with lazy loading, and due to that I need to execute javascript in PhantomJS browser to scroll down for last loaded element to load all and get all data.

After getting URL of specific LinkedIn user as follows:

driver.get("FullurlToLinkedInUser");

then I am able to receive last element on page successfully and I was trying this code (not my own, copied from: Scrolling with phantomJs Selenium ):

public synchronized WebDriver scrollToBottom(WebDriver driver, WebElement element,int time) throws InterruptedException {
    String oldpage="";
    String newpage="";


    do{
        oldpage=driver.getPageSource();
        ((JavascriptExecutor) driver)
                .executeScript("window.scrollTo(0, (document.body.scrollHeight))");
        this.wait(time);
        newpage=driver.getPageSource();
    }while(!oldpage.equals(newpage));
    return driver;
}

And after calling this function like this:

driver = scrollToBottom(driver,myWebElement, 2);

I will receive this error:

{"errorMessage":"
Refused to evaluate a string as JavaScript because 'unsafe- eval' is not an allowed source of script in the following Content Security Policy directive ... }

I am creating WebDriver as follows:

DesiredCapabilities caps = new DesiredCapabilities();
    ((DesiredCapabilities) caps).setJavascriptEnabled(true);
    ((DesiredCapabilities) caps).setCapability("takesScreenshot", true);
    ((DesiredCapabilities) caps).setCapability(
            PhantomJSDriverService.PHANTOMJS_EXECUTABLE_PATH_PROPERTY,
            "C:/Users/Bernad/Desktop/FIIT/ING/SEM1/AOVS/JAVA_selenium_phantomjs/phantomjs.exe"   //java web start / jnpl file...
            // "/Controller/phantomjs.exe"
    );

    //SET enabled javascript for php script on WEB page to transform it to picture:
    caps.setJavascriptEnabled(true);
    String [] phantomJsArgs = {"--web-security=no", "--ignore-ssl-errors=yes"};
    caps.setCapability(PhantomJSDriverService.PHANTOMJS_CLI_ARGS, phantomJsArgs);

    //CREATING WEB driver
    driver = new PhantomJSDriver(caps);

And I don't know if this is not connected to some previous error that I face to, that occurs in driver asynchronously after my login to LinkedIn and continuing to my connected people, these errors will not make PhantomJS to crash, but occurs, errors like:

[ERROR - 2017-11-20T11:25:47.737Z] Session [8dd6f810-cde5-11e7-9a5e-a3938000ab70] - page.onError - msg: TypeError: null is not an object (evaluating 'c.classList')

phantomjs://platform/console++.js:263 in error
[ERROR - 2017-11-20T11:25:47.737Z] Session [8dd6f810-cde5-11e7-9a5e-a3938000ab70] - page.onError - stack:
h (https://static.licdn.com/scds/concat/common/js?h=3kp2aedn5...:71 

(anonymous function) (https://static.licdn.com/scds/concat/common/js h=3kp2aedn5pmamdr4dk4n8atur-3ti5bgrnb6id...:71)
(anonymous function) (https://static.licdn.com/scds/concat/common/js?h=3kp2aedn5pmamdr4dk4n8atur-3ti5bgrnb6idjt...:71)
(anonymous function) (https://static.licdn.com/scds/concat/common/js?h=3kp2aedn5pmamdr4dk4n8atur-3ti5bgrnb6idjtk0w...:71)

phantomjs://platform/console++.js:263 in error
[ERROR - 2017-11-20T11:25:49.998Z] Session [8dd6f810-cde5-11e7-9a5e-a3938000ab70] - page.onError - msg: [object XMLHttpRequest]

I am sure that my WebElement sending to function exist. Please, could you give me a helping hand, because I didn't find similar solutions for JAVA implementation. Thanks in advance.

Marek Bernád
  • 481
  • 2
  • 8
  • 28
  • Does this question help you? https://stackoverflow.com/questions/39475547/phantomjs-automation-requires-unsafe-eval-to-work – Love Tätting Nov 20 '17 at 12:16
  • 1
    sorry but no, I have "--web-security=no" already tried, errors persist – Marek Bernád Nov 20 '17 at 13:56
  • Ok. I found solution for my problem, but not for those errors and why I can't run JS in PhantomJS from java. I received lazy loading elements by setting window size like this: driver.manage().window().setSize(new Dimension(4000,5000)); and afther that Thread.sleep(1000) and I got it – Marek Bernád Nov 20 '17 at 15:29
  • 1
    All solutions are good except the bad ones, as we say in Swedish – Love Tätting Nov 20 '17 at 16:42

0 Answers0