3

I have a large set of tests running on a site. When I run the test locally on Windows - they all pass on 100%. The test are designed and running on Google Chrome.

Now, we have started to run the tests on Linux via Jenkins jobs on headless mode. Some tests now fail on 0% or only pass on 20% or even 10%. In my code I'm finding elements by ID, xpath or css and simple click on them. And I use the WebDriverWait object for waiting - both for the element to be present and to be clickable.

Example of my code:

    WebDriverWait wait = new WebDriverWait(browser, secondsToWait);
    wait.until(ExpectedConditions.presenceOfElementLocated(By.id(elementID)));           
    lastFoundElement = wait.until(ExpectedConditions.elementToBeClickable(By.id(elementID)));
    clickLastFoundElement();

In my report I see mostly that the elements were not found and that I passed the timeout set in the wait object.

How to make headless tests be more stable?

Why the headless state causes so many problems?

Prophet
  • 32,350
  • 22
  • 54
  • 79
Tal Angel
  • 1,301
  • 3
  • 29
  • 63

3 Answers3

5

It is actually known issue to Selenium community that headless is not stable as it should be, read here about the issue more

Chrome runs very unstable when headless mode is activated. There are multiple different issues and bugs depending on: Chrome Version, ChromeDriver Version and the executed tests.

Issues and Fixes (occured so far):

Chrome does not start in headless mode

Exception:
No informative Error message. Only a timeout exception when navigate() is called on the driver:
org.openqa.selenium.TimeoutException: timeout

Fix:
options.addArguments("--proxy-server='direct://'");
options.addArguments("--proxy-bypass-list=*");

Chrome is very slow in headless mode

Fix:
options.addArguments("--proxy-server='direct://'");
options.addArguments("--proxy-bypass-list=*");

Chrome does not operate correctly after a certain time, when tests run very long resp. many actions are executed in one test session

Exception:
... Timed out receiving message from renderer: ...
Fix (not tested yet!):
options.addArguments("--disable-browser-side-navigation");
cruisepandey
  • 28,520
  • 6
  • 20
  • 38
  • will adding these arguments will cause the Selenium to always be stable in headless mode? What is the meaning of each argument? – Tal Angel Sep 06 '21 at 08:23
  • first make sure you have latest stable chromedriver, then make sure to set the size after that you should put waits, cause in headless mode HTTP response won't have heavy header size so it is very fast, to sync put proper delays. – cruisepandey Sep 06 '21 at 09:09
3

You possibly missing setting the headless window size.
Try this settings:

Map<String,String> prefs = new HashMap<>();
prefs.put("download.default_directory", downloadsPath); // Bypass default download directory in Chrome
prefs.put("safebrowsing.enabled", "false"); // Bypass warning message, keep file anyway (for .exe, .jar, etc.)
ChromeOptions opts = new ChromeOptions();
opts.setExperimentalOption("prefs", prefs);
opts.addArguments("--headless", "--disable-gpu", "--window-size=1920,1080","--ignore-certificate-errors","--no-sandbox", "--disable-dev-shm-usage");

driver = new ChromeDriver(opts);

Here I put much more setting used by me for the headless chrome.
I hope this will be useful for you.

Prophet
  • 32,350
  • 22
  • 54
  • 79
0

Please include screen size in the chrome option.

ChromeOptions options = new ChromeOptions();
options.addArguments(Arrays.asList("--window-position=0,0"));
options.addArguments(Arrays.asList("--window-size=1840,1080"));
Jayanth Bala
  • 758
  • 1
  • 5
  • 11