3

My Java Selenium Test case works fine when run using chrome driver, however the exact same code fails when run using HeadLess HTML driver.

I got the below java code using Katalon Selenium IDE.

import java.util.regex.Pattern;
import java.util.concurrent.TimeUnit;
import org.junit.*;
import static org.junit.Assert.*;
import static org.hamcrest.CoreMatchers.*;
import org.openqa.selenium.*;
import org.openqa.selenium.support.ui.Select;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.support.ui.*;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.htmlunit.HtmlUnitDriver;


public class QualysScan {
  private static ChromeDriver driver;


  public static void setUp() throws Exception {

    System.setProperty("webdriver.chrome.driver", "C:\\Users\\myuser\\Downloads\\chromedriver_win32\\chromedriver.exe");
    driver = new ChromeDriver();
    driver.manage().timeouts().implicitlyWait(60, TimeUnit.SECONDS);
  }


public static void testQualysScan() throws Exception {
    driver.get("https://qualysguard.myorg.com/fo/login.php?idm_key=saml2_78743hhh43");
    WebElement element = new WebDriverWait(driver, 20).until(ExpectedConditions.elementToBeClickable(By.xpath("//*[@id='userNameInput']")));
    driver.findElement(By.id("submitButton")).click();
    System.out.println("Title of the page is 8 -> " + driver.getTitle());

    WebElement element1 = new WebDriverWait(driver, 50).until(ExpectedConditions.elementToBeClickable(By.xpath("//*[@id='ext-gen14']")));
    System.out.println("Title of the page is 9 -> " + driver.getTitle());
}

This test-case works fine when run using chrome browser plugin driver as in the above code.

However, when I switch to headless HTML driver the same code fails with the below error:

Code changes for HeadLess are as below:

public class QualysScan {
 private static WebDriver driver;

  public static void setUp() throws Exception {
    driver = new org.openqa.selenium.htmlunit.HtmlUnitDriver();
    driver.manage().timeouts().implicitlyWait(60, TimeUnit.SECONDS);
  }

However, running the exported java code gives me the below error:

Output:

Title of the page is 8 -> Sign In
Exception in thread "main" org.openqa.selenium.TimeoutException: Expected condition failed: waiting for element to be clickable: By.xpath: //*[@id='ext-gen14'] (tried for 50 second(s) with 500 milliseconds interval)
at org.openqa.selenium.support.ui.WebDriverWait.timeoutException(WebDriverWait.java:95)
at org.openqa.selenium.support.ui.FluentWait.until(FluentWait.java:272)
at pack.QualysScan.testQualysScan(QualysScan.java:110)
at pack.QualysScan.main(QualysScan.java:247)
Caused by: org.openqa.selenium.NoSuchElementException: Unable to locate a node using //*[@id='ext-gen14']
For documentation on this error, please visit: http://seleniumhq.org/exceptions/no_such_element.html
Build info: version: '3.141.0', revision: '2ecb7d9a', time: '2018-10-31T20:09:30'
System info: host: 'MYHOST7', ip: '10.95.112.212', os.name: 'Windows 7', os.arch: 'amd64', os.version: '6.1', java.version: '1.8.0_121'
Driver info: driver.version: HtmlUnitDriver
at org.openqa.selenium.htmlunit.HtmlUnitDriver.findElementByXPath(HtmlUnitDriver.java:1421)
at org.openqa.selenium.By$ByXPath.findElement(By.java:353)
at org.openqa.selenium.htmlunit.HtmlUnitDriver$4.call(HtmlUnitDriver.java:2042)
at org.openqa.selenium.htmlunit.HtmlUnitDriver$4.call(HtmlUnitDriver.java:2038)
at org.openqa.selenium.htmlunit.HtmlUnitDriver.implicitlyWaitFor(HtmlUnitDriver.java:1673)
at org.openqa.selenium.htmlunit.HtmlUnitDriver.findElement(HtmlUnitDriver.java:2038)
at org.openqa.selenium.htmlunit.HtmlUnitDriver.findElement(HtmlUnitDriver.java:810)
at org.openqa.selenium.support.ui.ExpectedConditions$7.apply(ExpectedConditions.java:205)
at org.openqa.selenium.support.ui.ExpectedConditions$7.apply(ExpectedConditions.java:201)
at org.openqa.selenium.support.ui.ExpectedConditions$22.apply(ExpectedConditions.java:641)
at org.openqa.selenium.support.ui.ExpectedConditions$22.apply(ExpectedConditions.java:638)
at org.openqa.selenium.support.ui.FluentWait.until(FluentWait.java:249)
... 2 more

I tried to find "ext-gen14" in view source of the page but could not find it. I understand that my page has javascript code.

I also find a few clickable items id using xpath finder plugin as below:

/html/body/div[1]/div[5]/div[2]/a
/html/body/div[1]/div[5]/div[3]/a
/html/body/div[1]/div[5]/div[6]/a

These are buttons and thier entry in the view source shows as below:

<div class="clear"></div>
<div id="top_modules_bar">
  <div class="module-tabs-tab module-tabs-tab-selected">
    <a onclick="javascript: showProcessing('Dashboard');" href="/fo/compliance/index.php?skip=1" class="module-link">Dashboard</a>
  </div>
  <div class="module-tabs-tab">
    <a onclick="javascript: showProcessing('Policies');" href="/fo/tools/module_landing.php?module=prod_policies" class="module-link">Policies</a>
  </div>
  <div class="module-tabs-tab">
    <a onclick="javascript: showProcessing('Scans');" href="/fo/tools/module_landing.php?module=prod_scans" class="module-link">Scans</a>
  </div>
  <div class="module-tabs-tab">
    <a onclick="javascript: showProcessing('Reports');" href="/fo/tools/module_landing.php?module=prod_reports" class="module-link">Reports</a>
  </div>
  <div class="module-tabs-tab">
    <a onclick="javascript: showProcessing('Exceptions');" href="/fo/tools/module_landing.php?module=prod_exceptions" class="module-link">Exceptions</a>
  </div>
  <div class="module-tabs-tab">
    <a onclick="javascript: showProcessing('Assets');" href="/fo/tools/module_landing.php?module=prod_assets" class="module-link">Assets</a>
  </div>
  <div class="module-tabs-tab">
    <a onclick="javascript: showProcessing('Users');" href="/fo/tools/module_landing.php?module=prod_users" class="module-link">Users</a>
  </div>
  <div class="clear"></div>
</div>

Can you please suggest how can i make sure that any of these items are loaded properly before i fire the click ?

I was also interested to know as to why would my java code work fine when run using chrome driver and fail when run as a headless HTML driver ?

Kindly suggest.

Ashar
  • 2,942
  • 10
  • 58
  • 122
  • For running the test in Headless mode .Rather than using HTMLUnitDriver you can use Chrome in headless mode. `ChromeOptions options = new ChromeOptions(); options.setHeadless(true); driver = new ChromeDriver(options);` – Rahul L Dec 17 '19 at 09:46
  • Can i use Firefox in headless mode. Please suggest the same for Firefox. – Ashar Dec 17 '19 at 10:38
  • Yes you can use firefox in headless mode .FirefoxOptions options = new FirefoxOptions(); options.setHeadless(true); WebDriver driver = new FirefoxDriver(options); – Rahul L Dec 17 '19 at 10:40
  • @rahul i was trying Chrome headless and i was able to bypass "ext-gen14" but got error down the line. Error is:Exception in thread "main" org.openqa.selenium.NoSuchElementException: no such element: Unable to locate element: {"method":"css selector","selector":"#ext\-gen117"} (Session info: headless chrome=77.0.3865.75) – Ashar Dec 17 '19 at 10:53
  • Your ID must be dynamic.Can you post that HTML you are getting exception. – KunduK Dec 17 '19 at 11:21
  • @kunduk it's too big to post here. Will try to post. – Ashar Dec 17 '19 at 11:34

0 Answers0