Stack Overflow
Stack Overflow

Questions tagged [htmlunit]

HtmlUnit is a "headless browser". Which means that there is no browser GUI and it does no rendering. Though it has a CSS and JS engine to simulate a real browser. Primary purpose is testing and information extraction.

HtmlUnit is a "GUI-Less browser for Java programs". It models HTML documents and provides an API that allows you to invoke pages, fill out forms, click links, etc... just like you do in your "normal" browser.

It has fairly good JavaScript support (which is constantly improving) and is able to work even with quite complex AJAX libraries, simulating Chrome, Firefox or Internet Explorer depending on the configuration used.

It is typically used for testing purposes or to retrieve information from web sites.

HtmlUnit is not a generic unit testing framework. It is specifically a way to simulate a browser for testing purposes and is intended to be used within another testing framework such as JUnit or TestNG.

HtmlUnit is used as the underlying "browser" by different Open Source tools like Canoo WebTest, JWebUnit, Selenium WebDriver, JSFUnit, Celerity, ...

HtmlUnit was originally written by Mike Bowler of Gargoyle Software and is released under the Apache 2 license.

Useful links

1835 questions
0
votes
1 answer

URL is not resolved in ubuntu 14.04

I have a cron job schedule in my Ubuntu or Runnable JAR running. I have received an exception on my string URL but on my Windows development it is OK. it could not resolve the correct string url please check below java.io.FileNotFoundException:…
Tiborsio_
  • 63
  • 1
  • 13
0
votes
1 answer

HtmlUnit won't open a link unless I manually tell it to

This works: HtmlPage page = (HtmlPage) browser.getPage("http://www.somewebsite.com/viewprofile.aspx?profile_id=107992814") However if I put the URL in a variable like this: String userPage = "http://www.somewebsite.com/" +…
Sentinel
  • 441
  • 1
  • 6
  • 25
0
votes
1 answer

HtmlUnit and decrypting a span element

I am trying to scrape a telephone number from a website. When i inspect the telephone number from the second entry, the inspector in Chrome gives me the following result: (012) 34 56 78
user5551274
0
votes
0 answers

Is it possible to set cookies in htmlunit/webclient

I have this code but I couldn't find best solution for this. I have no idea on the cookie side. here is the code & Warning Cookie. I'm trying to login for the website after that I use getPage to download a file then write it into my…
Tiborsio_
  • 63
  • 1
  • 13
0
votes
3 answers

Parsing HTML doc without using tag or any other selector in Java

I have a HTML page, lets say http://www.crisil.com/Ratings/RatingList/RatingDocs/_G_Telecom_Infra_India_Private_Limited_August_28_2015_RR.html I want to parse About the Company paragraph and the below table without using any kind of selector or…
spondon majumdar
  • 11
  • 4
0
votes
1 answer

slf4j error with Htmlunit Maven dependency

With the following Maven dependency for Htmlunit, net.sourceforge.htmlunit htmlunit 2.18 I am getting the following error, SLF4J: Class path…
smart987
  • 834
  • 2
  • 14
  • 34
0
votes
0 answers

Populate values in drop down list based on the selection in the above drop down list in HtmlUnit

I am working on scraping of a website through HtmlUnit. I have 2 drop down lists in it. The 2 drop down lists are linked such that the selection of drop down list 1 option makes a JavaScript call on onChange(). Based on this selection, the values in…
shubh
  • 253
  • 2
  • 3
  • 9
0
votes
0 answers

HtmlUnit + java: cannot open SSL site. SSLException: Received fatal alert: illegal_parameter

I've had some experience with HtmlUnit (2.18 + openjdk8) on other tasks, but now struggling to even enter one site (it is "RSA security console", I cannot make it public). Real browsers behave differently: IE11 works flawlessly. Chromium doesn't…
Oleg Gritsak
  • 548
  • 7
  • 26
0
votes
0 answers

Htmlunit auto-login to google

I am using htmlunit to automate logging on to a server with google (the red "Sign in with Google button"). This redirects https://accounts.google.com/ServiceLogin, where I must write the email and password attributes every time. Compared to if I go…
Karl
  • 26
  • 3
0
votes
2 answers

Error using HtmlUnit 2.18 for crawl webpages

I have the following code: WebClient webClient = new WebClient(BrowserVersion.getDefault()); HtmlPage page; List anchor=new ArrayList(); try { System.out.println("Querying"); page =…
TheGaME
  • 443
  • 2
  • 8
  • 21
0
votes
1 answer

Grab an html file with executed Javascripts in Java

I recently found out how to grab a html code using java. Therefore i progammed the following method: public String htmlToString(String urlString){ //returns the html code of the given website into a string //if something doesn't work…
oRookie
  • 265
  • 2
  • 4
  • 13
0
votes
1 answer

Login and get Webpage using API HtmlUnit in Java

I'm trying to get a webpage. I'm getting the form, the textinput, the checkbox and the submit button so I can fill these ones trough java code. First of all I'm getting these warnings (I presume that ScriptEngine fail to load some scripts): oct…
Saverio Java
  • 78
  • 11
0
votes
0 answers

download excel file using htmlunit

I am trying to download xls file from a website using Htmlunit. When I click the link to download the file, I get a javascript confirm box. I tried multiple options but not able to find the slution: ConfirmHandler okHandler = new…
Bhoomi Akhani
  • 25
  • 1
  • 5
0
votes
0 answers

Javascript enabling is not working in chrome in htmlunitdriver

I am trying to get data from e-commerce sites by crawling using jsoup. Since I am getting dynamic contents i am using HtmlUnitDriver. But it is returning only empty tags as given below. I have also enabled javascript by giving the following…
Avles
  • 15
  • 2
  • 9
0
votes
0 answers

Java code doesn't work as expected when used as library

I made a simple program to login to the instagram web site. My original code works in a java main class but now i stumbled with something strange, when i'm putting the code, i'm using it in another class, so i can load and use it in other classes…
Progs
  • 1,059
  • 7
  • 27
  • 63
1 2 3
…
99 100