1

Im in the process of learning how to use Selenium to automate we browsers and working my way through an online course.

Im currently having some problems trying to run a headless browser test using the HtmlUnitDriver

I have a simple test that should load a web page and assert the title, code as follows.

package com.seleniumsimplified.webdriver;

import org.junit.Test;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.htmlunit.HtmlUnitDriver;

import static org.junit.Assert.assertTrue;


public class MyFirstTest {

@Test
public void driverIsTheKing(){


    WebDriver driver = new HtmlUnitDriver();

    driver.get("http://compendiumdev.co.uk/selenium");
    assertTrue(driver.getTitle().startsWith("Selenium Simplified"));



  }
}

On attempting to run the test, the test does not appear to run properly, it takes around 4 minuites after which time i receive the following error which im not sure on how to resolve:

"C:\Program Files\Java\jdk1.8.0_131\bin\java" -ea -Didea.launcher.port=7532 "-Didea.launcher.bin.path=C:\Program Files (x86)\JetBrains\IntelliJ IDEA Community` Edition 2016.2\bin" -Didea.junit.sm_runner -Dfile.encoding=UTF-8 -classpath "C:\Program Files (x86)\JetBrains\IntelliJ IDEA Community Edition 2016.2\lib\idea_rt.jar;C:\Program Files (x86)\JetBrains\IntelliJ IDEA Community Edition 2016.2\plugins\junit\lib\junit-rt.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\charsets.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\deploy.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\access-bridge-64.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\cldrdata.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\dnsns.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\jaccess.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\jfxrt.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\localedata.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\nashorn.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\sunec.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\sunjce_provider.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\sunmscapi.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\sunpkcs11.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\ext\zipfs.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\javaws.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\jce.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\jfr.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\jfxswt.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\jsse.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\management-agent.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\plugin.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\resources.jar;C:\Program Files\Java\jdk1.8.0_131\jre\lib\rt.jar;C:\Users\ra.r.hope\IdeaProjects\webdriverbasics\target\test-classes;C:\Users\robert.hope\.m2\repository\junit\junit\4.12\junit-4.12.jar;C:\Users\robert.hope\.m2\repository\org\hamcrest\hamcrest-core\1.3\hamcrest-core-1.3.jar;C:\Users\robert.hope\.m2\repository\org\seleniumhq\selenium\selenium-java\3.5.3\selenium-java-3.5.3.jar;C:\Users\robert.hope\.m2\repository\org\seleniumhq\selenium\selenium-api\3.5.3\selenium-api-3.5.3.jar;C:\Users\robert.hope\.m2\repository\org\seleniumhq\selenium\selenium-chrome-driver\3.5.3\selenium-chrome-driver-3.5.3.jar;C:\Users\robert.hope\.m2\repository\org\seleniumhq\selenium\selenium-edge-driver\3.5.3\selenium-edge-driver-3.5.3.jar;C:\Users\robert.hope\.m2\repository\org\seleniumhq\selenium\selenium-firefox-driver\3.5.3\selenium-firefox-driver-3.5.3.jar;C:\Users\robert.hope\.m2\repository\org\seleniumhq\selenium\selenium-ie-driver\3.5.3\selenium-ie-driver-3.5.3.jar;C:\Users\robert.hope\.m2\repository\org\seleniumhq\selenium\selenium-opera-driver\3.5.3\selenium-opera-driver-3.5.3.jar;C:\Users\robert.hope\.m2\repository\org\seleniumhq\selenium\selenium-remote-driver\3.5.3\selenium-remote-driver-3.5.3.jar;C:\Users\robert.hope\.m2\repository\org\seleniumhq\selenium\selenium-safari-driver\3.5.3\selenium-safari-driver-3.5.3.jar;C:\Users\robert.hope\.m2\repository\org\seleniumhq\selenium\selenium-support\3.5.3\selenium-support-3.5.3.jar;C:\Users\robert.hope\.m2\repository\cglib\cglib-nodep\3.2.4\cglib-nodep-3.2.4.jar;C:\Users\robert.hope\.m2\repository\org\apache\commons\commons-exec\1.3\commons-exec-1.3.jar;C:\Users\robert.hope\.m2\repository\org\apache\commons\commons-lang3\3.5\commons-lang3-3.5.jar;C:\Users\robert.hope\.m2\repository\commons-codec\commons-codec\1.10\commons-codec-1.10.jar;C:\Users\robert.hope\.m2\repository\commons-io\commons-io\2.5\commons-io-2.5.jar;C:\Users\robert.hope\.m2\repository\commons-logging\commons-logging\1.2\commons-logging-1.2.jar;C:\Users\robert.hope\.m2\repository\org\w3c\css\sac\1.3\sac-1.3.jar;C:\Users\robert.hope\.m2\repository\net\sourceforge\cssparser\cssparser\0.9.23\cssparser-0.9.23.jar;C:\Users\robert.hope\.m2\repository\com\google\code\gson\gson\2.8.0\gson-2.8.0.jar;C:\Users\robert.hope\.m2\repository\com\google\guava\guava\23.0\guava-23.0.jar;C:\Users\robert.hope\.m2\repository\com\google\code\findbugs\jsr305\1.3.9\jsr305-1.3.9.jar;C:\Users\robert.hope\.m2\repository\com\google\errorprone\error_prone_annotations\2.0.18\error_prone_annotations-2.0.18.jar;C:\Users\robert.hope\.m2\repository\com\google\j2objc\j2objc-annotations\1.1\j2objc-annotations-1.1.jar;C:\Users\robert.hope\.m2\repository\org\codehaus\mojo\animal-sniffer-annotations\1.14\animal-sniffer-annotations-1.14.jar;C:\Users\robert.hope\.m2\repository\net\sourceforge\htmlunit\htmlunit\2.27\htmlunit-2.27.jar;C:\Users\robert.hope\.m2\repository\net\sourceforge\htmlunit\htmlunit-core-js\2.27\htmlunit-core-js-2.27.jar;C:\Users\robert.hope\.m2\repository\net\sourceforge\htmlunit\neko-htmlunit\2.27\neko-htmlunit-2.27.jar;C:\Users\robert.hope\.m2\repository\org\apache\httpcomponents\httpclient\4.5.3\httpclient-4.5.3.jar;C:\Users\robert.hope\.m2\repository\org\apache\httpcomponents\httpcore\4.4.6\httpcore-4.4.6.jar;C:\Users\robert.hope\.m2\repository\org\apache\httpcomponents\httpmime\4.5.3\httpmime-4.5.3.jar;C:\Users\robert.hope\.m2\repository\org\eclipse\jetty\jetty-client\9.4.5.v20170502\jetty-client-9.4.5.v20170502.jar;C:\Users\robert.hope\.m2\repository\org\eclipse\jetty\jetty-http\9.4.5.v20170502\jetty-http-9.4.5.v20170502.jar;C:\Users\robert.hope\.m2\repository\org\eclipse\jetty\jetty-io\9.4.5.v20170502\jetty-io-9.4.5.v20170502.jar;C:\Users\robert.hope\.m2\repository\org\eclipse\jetty\jetty-util\9.4.5.v20170502\jetty-util-9.4.5.v20170502.jar;C:\Users\robert.hope\.m2\repository\net\java\dev\jna\jna\4.1.0\jna-4.1.0.jar;C:\Users\robert.hope\.m2\repository\net\java\dev\jna\jna-platform\4.1.0\jna-platform-4.1.0.jar;C:\Users\robert.hope\.m2\repository\com\codeborne\phantomjsdriver\1.4.0\phantomjsdriver-1.4.0.jar;C:\Users\robert.hope\.m2\repository\javax\servlet\javax.servlet-api\3.1.0\javax.servlet-api-3.1.0.jar;C:\Users\robert.hope\.m2\repository\org\eclipse\jetty\websocket\websocket-api\9.4.5.v20170502\websocket-api-9.4.5.v20170502.jar;C:\Users\robert.hope\.m2\repository\org\eclipse\jetty\websocket\websocket-client\9.4.5.v20170502\websocket-client-9.4.5.v20170502.jar;C:\Users\robert.hope\.m2\repository\org\eclipse\jetty\websocket\websocket-common\9.4.5.v20170502\websocket-common-9.4.5.v20170502.jar;C:\Users\robert.hope\.m2\repository\xalan\serializer\2.7.2\serializer-2.7.2.jar;C:\Users\robert.hope\.m2\repository\xalan\xalan\2.7.2\xalan-2.7.2.jar;C:\Users\robert.hope\.m2\repository\xerces\xercesImpl\2.11.0\xercesImpl-2.11.0.jar;C:\Users\robert.hope\.m2\repository\xml-apis\xml-apis\1.4.01\xml-apis-1.4.01.jar;C:\Users\robert.hope\.m2\repository\org\seleniumhq\selenium\htmlunit-driver\2.27\htmlunit-driver-2.27.jar" com.intellij.rt.execution.application.AppMain com.intellij.rt.execution.junit.JUnitStarter -ideVersion5 com.seleniumsimplified.webdriver.MyFirstTest,driverIsTheKing
Sep 13, 2017 8:24:34 AM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (java.net.SocketException) caught when processing request to {s}->https://compendiumdev.co.uk:443: Connection reset
Sep 13, 2017 8:24:34 AM org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {s}->https://compendiumdev.co.uk:443
Sep 13, 2017 8:25:34 AM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (java.net.SocketException) caught when processing request to {s}->https://compendiumdev.co.uk:443: Connection reset
Sep 13, 2017 8:25:34 AM org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {s}->https://compendiumdev.co.uk:443
Sep 13, 2017 8:26:43 AM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (java.net.SocketException) caught when processing request to {s}->https://compendiumdev.co.uk:443: Connection reset
Sep 13, 2017 8:26:43 AM org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {s}->https://compendiumdev.co.uk:443

org.openqa.selenium.WebDriverException: java.net.SocketException: Connection reset
Build info: version: '3.5.3', revision: 'a88d25fe6b', time: '2017-08-29T12:42:44.417Z'
System info: host: 'LT-LGC07999', ip: '10.44.22.14', os.name: 'Windows 7', os.arch: 'amd64', os.version: '6.1', java.version: '1.8.0_131'
Driver info: driver.version: HtmlUnitDriver

    at org.openqa.selenium.htmlunit.HtmlUnitDriver.get(HtmlUnitDriver.java:686)
    at org.openqa.selenium.htmlunit.HtmlUnitDriver.lambda$8(HtmlUnitDriver.java:655)
    at org.openqa.selenium.htmlunit.HtmlUnitDriver.lambda$0(HtmlUnitDriver.java:412)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketException: Connection reset
    at java.net.SocketInputStream.read(SocketInputStream.java:210)
    at java.net.SocketInputStream.read(SocketInputStream.java:141)
    at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
    at sun.security.ssl.InputRecord.read(InputRecord.java:503)
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
    at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1769)
    at sun.security.ssl.HandshakeOutStream.flush(HandshakeOutStream.java:124)
    at sun.security.ssl.Handshaker.kickstart(Handshaker.java:1067)
    at sun.security.ssl.SSLSocketImpl.kickstartHandshake(SSLSocketImpl.java:1487)
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1351)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387)
    at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396)
    at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355)
    at com.gargoylesoftware.htmlunit.httpclient.HtmlUnitSSLConnectionSocketFactory.connectSocket(HtmlUnitSSLConnectionSocketFactory.java:186)
    at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
    at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359)
    at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381)
    at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237)
    at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
    at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
    at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
    at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72)
    at com.gargoylesoftware.htmlunit.HttpWebConnection.getResponse(HttpWebConnection.java:194)
    at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseFromWebConnection(WebClient.java:1379)
    at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseFromWebConnection(WebClient.java:1429)
    at com.gargoylesoftware.htmlunit.WebClient.loadWebResponse(WebClient.java:1298)
    at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:392)
    at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:313)
    at org.openqa.selenium.htmlunit.HtmlUnitDriver.get(HtmlUnitDriver.java:668)
    ... 3 more

I did some research online and found out that javascript is disabled by default when using the HTMLUnitDriver, however im not sure if this is significant to my problem

I experimented with enabling java script by setting the value to true when creating a new HtmlUnitDriver

WebDriver driver = new HtmlUnitDriver(true);

I tried doing this and re running the code but unfortunatley i reveive the same result when running the test......

Has anyone come across this type of issue before and is able to provide any guidance on how i can get the test to run.

For reference im using Selenium 3.53 junit 4.12 and htmlUintDriver 2.27

Additonaly i can run the same test using the chromeDriver or the firefoxDriver and the test runs and passes correctly

Thanks

bobe_b
  • 187
  • 1
  • 2
  • 21
  • Use headless chrome instead of this – iamsankalp89 Sep 13 '17 at 12:02
  • this is probably an option but im trying to stick with the course instructions for the time being for the purposes of learning each approach. Hence i would like to stick with the htmlUnitDriver for the time being if possible – bobe_b Sep 13 '17 at 13:50

1 Answers1

0

HtmlUnitDriver was a part of Selenium version 2.53. If you are using Selenium 2.52 or earlier you don't need to download and install HtmlUnitDriver, it is already there.

But still you want to use it download it from here

https://github.com/SeleniumHQ/htmlunit-driver/releases

From 2.53 version, you need to add it separately

iamsankalp89
  • 4,607
  • 2
  • 15
  • 36
  • Im using selenium 3.5.3 and have added the HtmlUnitDriver dependency to my POM file like this: org.seleniumhq.selenium htmlunit-driver 2.27 Does this look correct to you? – bobe_b Sep 13 '17 at 11:29
  • I am run your code on 2.53 and 3.5 with HtmlUnitdriver – iamsankalp89 Sep 13 '17 at 12:01
  • when you mention downloading the jar I take it you are referring to the htmluintdriver jar? do i just need to download and run it or does it need to be stored somewhere specific? sorry for the newb additional q's. Im sure il pick this stuf up with time and practice! – bobe_b Sep 13 '17 at 13:47
  • No issue , i Will suggest use Headless chrome, it is better than HtmlUnitDriver – iamsankalp89 Sep 13 '17 at 13:50
  • Im still facing the same issue whereby each time i try to run the code i receive a series of I/O exception messages followed by the other messages as detailed in my original post. Thanks for investigating this – bobe_b Sep 13 '17 at 14:03
  • Hey iamsankalp89, please can you let me know the following info on when you successfully ran the test using a version of selenium above 2.5.3: Junit version, HtmlUnitDriver version, Maven version, selenium version. I suspect i might have a combination that may have some issues so id like to try running the test on the same versions as yourself to see if it works – bobe_b Sep 14 '17 at 08:44
  • @bobe_b i used selenium jar 2.53. – iamsankalp89 Sep 14 '17 at 08:46
  • Your code also working in my seleniium 3.5. i am sure your configuration have some issue – iamsankalp89 Sep 14 '17 at 08:56
  • thanks mate, i will go back to the drawing board and see if i can work out what the problem is – bobe_b Sep 14 '17 at 13:56
  • org.seleniumhq.selenium selenium-htmlunit-driver 2.52.0 USe this dependancy – iamsankalp89 Sep 15 '17 at 09:00