Stack Overflow
Stack Overflow

Questions tagged [htmlunit]

HtmlUnit is a "headless browser". Which means that there is no browser GUI and it does no rendering. Though it has a CSS and JS engine to simulate a real browser. Primary purpose is testing and information extraction.

HtmlUnit is a "GUI-Less browser for Java programs". It models HTML documents and provides an API that allows you to invoke pages, fill out forms, click links, etc... just like you do in your "normal" browser.

It has fairly good JavaScript support (which is constantly improving) and is able to work even with quite complex AJAX libraries, simulating Chrome, Firefox or Internet Explorer depending on the configuration used.

It is typically used for testing purposes or to retrieve information from web sites.

HtmlUnit is not a generic unit testing framework. It is specifically a way to simulate a browser for testing purposes and is intended to be used within another testing framework such as JUnit or TestNG.

HtmlUnit is used as the underlying "browser" by different Open Source tools like Canoo WebTest, JWebUnit, Selenium WebDriver, JSFUnit, Celerity, ...

HtmlUnit was originally written by Mike Bowler of Gargoyle Software and is released under the Apache 2 license.

Useful links

1835 questions
5
votes
1 answer

HTMLUNIT v. 2.15 Bug Thread deadlock

I'm using htmlunit (version 2.15); I noticed a memory leak issue with the WebClient class: all webClient instances are not cleaned by garbage collector. The issue seems caused by threads deadlock: …
Azincourt
  • 935
  • 5
  • 13
  • 29
5
votes
1 answer

How to post HEAD request with HtmlUnit?

How can I use existing WebClient to post a custom HEAD request? I.e. there is no form on the page, I want to set request properties manually.
Fluffy
  • 27,504
  • 41
  • 151
  • 234
5
votes
0 answers

HTMLUnit not able to provide updated page on scroll

I am trying to get data from a page that dynamically appends data on load. Specifications: HTMLUnit : version : 2.14 But I am not able to get the new page after scroll. I tried using various browser versions and all possible code changes. It will be…
ReeniMathew
  • 69
  • 5
5
votes
3 answers

How to unit test my HtmlUnit code with an xml file

I am writing an application that uses HtmlUnit to screen scrape some data. The logic of which fields come from which parts of the page, and the XPath to retrieve them is getting a bit complicated, so before I refactor I want to write some simple…
Matt
  • 3,303
  • 5
  • 31
  • 53
5
votes
2 answers

Issue Using Scanner in Java

I am writing a program that needs to read data from an input text file, and save a variables upon going through the date. I am using Htmlunit, and am running into the error: com.gargoylesoftware.htmlunit.ScriptException: Exception invoking open …
Ctech45
  • 496
  • 9
  • 17
5
votes
1 answer

angular-mock.js conflicts with htmlunit? How to test angular application using angular-mock.js in maven-jasmine

I'm new to maven and jasmine, and I have a problem setting up maven to run unit tests on an angular application. I get into trouble as soon as I add angular-mocks.js to the test/javascript/lib folder. mvn test throws a htmlunit.ScriptException just…
c.salaets
  • 53
  • 4
5
votes
1 answer

htmlunit: return a completely loaded page

I am using HtmlUnit library for Java to manipulate websites programmatically. I can't find the working solution to my problem: How to determine that all AJAX calls are finished and return a completely loaded webpage? Here's what I have…
justasd
  • 401
  • 1
  • 12
  • 26
5
votes
2 answers

java htmlunit failing to load Javascript

What is causing this error, and how do I fix it? WebClient webClient = new WebClient(); HtmlPage page = webClient.getPage("http://stackoverflow.com"); Error message May 25, 2013 10:34:12 AM com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl…
Spik330
  • 502
  • 4
  • 6
  • 19
5
votes
3 answers

Selenium testing machine

--- TL;DR At this point I suggest everyone to tied their Continuous Integration server/service to https://ghostinspector.com/ OLD QUESTION after three days googling and testing I gave up, and I need help. My objective is allow my co-workers to…
Davis Peixoto
  • 5,695
  • 2
  • 23
  • 35
5
votes
4 answers

how to use htmlunit with my android project

I have downloaded htmlunit 2.11 zip. i have extract it. then i have tried to paste them in my project's libs folder. from the libs folder i have added them in build path. then i get this error, while i was trying to run my app conversion to dalvik…
Shoshi
  • 2,254
  • 1
  • 29
  • 43
5
votes
1 answer

HtmlUnit Only Displays Host HTML Page for GWT App

I am using HtmlUnit API to add crawler support to my GWT app as follows: PrintWriter out = null; try { resp.setCharacterEncoding(CHAR_ENCODING); resp.setContentType("text/html"); url = buildUrl(req); out = resp.getWriter(); …
Peter
  • 1,720
  • 3
  • 18
  • 30
5
votes
2 answers

HtmlUnit Session Management

I'm trying to login to Facebook page using HtmlUnit and view its HTML content. I'm trying to fill up the login credentials through HtmlUnit but I don't see the session being carried when the submit button is clicked. Couldnt find much content on…
rishiehari
  • 384
  • 5
  • 13
5
votes
3 answers

HtmlUnit ScriptException errors

I am using HtmlUnitDriver,& here is my code. HtmlUnitDriver driver = new HtmlUnitDriver(true); driver.get("some url here"); I am getting following Exception: Caused by: com.gargoylesoftware.htmlunit.ScriptException: Wrapped…
dhananjay
  • 321
  • 2
  • 11
  • 20
5
votes
1 answer

Getting Jsoup to support dynamically generated html by JavaScript

right now I'm working on a webcrawler. This one should parse some specific sites and give me an output into an xml-file. Up to this point, it's no problem. The Crawler works and you can customize it realy quickly via a cfg-file. I use Jsoup to parse…
Ogofo
  • 356
  • 2
  • 6
  • 13
5
votes
2 answers

Clicking on javascript anchor with htmlunit does not seem to work

I am using htmlunit to automatically go through a website. Here is the problem: I want to click on an anchor in order to display a new page of a given table. Here is the anchor: 7 Here is…
user1190900
  • 85
  • 1
  • 6
1 2 3
…
99 100