selenium tests fail against headless chrome

Question

I am trying to get my selenium test automation to run against headless chrome so that I can move it to TeamCity. I have not had any luck. When I run it, Chrome does appear to run headlessly (no browser pops up), but I get a NoSuchElementException. The automation works as expected when run non-headlessly. A snapshot taken just shows a white rectangle.

I have researched this issue extensively, but I have not been able to find a solution that works for me. It appears that the issue was reported in https://bugs.chromium.org/p/chromedriver/issues/detail?id=476, but it's marked fixed. I think the problem might be the wrong chromedriver, or maybe the wrong chromedriver/selenium combination, but I've tried all sorts of combinations and no love.

I am using:

selenium-java 3.6.0
chromedriver 2.33.506120
Windows 7 Enterprise Service Pack1, 64-bit

My code is:

...
ChromeOptions headlessOptions = new ChromeOptions();
headlessOptions.addArguments("--start-maximized");
headlessOptions.addArguments("--headless");
driver = new ChromeDriver(headlessOptions);
driver.get(url);
WebElement usernameTextfield = driver.findElement(By.cssSelector(".input.username"));
...

And the output is:

Starting ChromeDriver 2.33.506120 (e3e53437346286c0bc2d2dc9aa4915ba81d9023f) on port 41402
Only local connections are allowed.
Nov 01, 2017 10:22:51 AM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: OSS

org.openqa.selenium.NoSuchElementException: no such element: Unable to locate element: {"method":"css selector","selector":".input.username"}
  (Session info: headless chrome=62.0.3202.75)
  (Driver info: chromedriver=2.33.506120 (e3e53437346286c0bc2d2dc9aa4915ba81d9023f),platform=Windows NT 6.1.7601 SP1 x86_64) (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 0 milliseconds

This is preventing me from being able to include my test automation as part of our CI, so any help would be very much appreciated.

Try debugging the site in headless browser and see if the element is actually missing. See this thread for help on debugging https://stackoverflow.com/questions/46017982/debugging-with-headless-browser/46018351#46018351 — Tarun Lalwani, Nov 01 '17 at 18:57
"Chrome does appear to run headlessly (no browser pops up)" - what is your expectation of a headless browser?!?! What is `url`? Is it https, and is it using self-signed certificate? — SiKing, Nov 01 '17 at 19:01
@TarunLalwani, this was a great suggestion. Unfortunately, it confirmed what the screenshot was telling me - yes, the element is missing, because there's nothing there except a blank screen. Which leads me back to thinking there's a problem with how I'm using selenium/chromedriver/headless. — dstaraster, Nov 01 '17 at 22:31
@SiKing My expectation of a headless browser is that it doesn't pop up on the monitor. I think that you may have mentally inserted "not" between "does" & "appear" in "Chrome does appear to run headlessly" :-) The URL is https, and it does use a self-signed certificate, but we have a setting on my test server that ignores checking the certificate. — dstaraster, Nov 01 '17 at 22:36
Checking the cert is done by the browser, not the server. If you take a screenshot in your (failed) test, you will get only a blank page (as you have confirmed). Chrome-headless currently does **not** allow you to ignore bad certs. Chrome-GUI does. See https://bugs.chromium.org/p/chromium/issues/detail?id=721739 — SiKing, Nov 01 '17 at 23:30
And you are correct about the invisible "not". ;) Sorry, my bad. — SiKing, Nov 01 '17 at 23:31
@SiKing Thank you. I'm pretty sure that you nailed the issue. I'll talk with one of our devs. — dstaraster, Nov 02 '17 at 15:46

score 10 · Answer 1 · answered Aug 06 '18 at 15:48

This is what worked for me:

var chromeOptions = new ChromeOptions();                        
chromeOptions.AddArguments("--headless");
chromeOptions.AddArguments("--disable-gpu");
chromeOptions.AddArguments("--window-size=1280,800");
chromeOptions.AddArguments("--allow-insecure-localhost");

//specifically this line here :)
chromeOptions.AddAdditionalCapability("acceptInsecureCerts", true, true);

Found from https://bugs.chromium.org/p/chromium/issues/detail?id=721739

score 8 · Answer 2 · edited Jan 04 '18 at 01:21

8

I had the same issue, the local server was using selfsigned certificate, here is the combination that worked for me:

ChromeOptions options = new ChromeOptions();
options.addArguments("--headless");
options.addArguments("--disable-gpu");
options.addArguments("--no-sandbox");
options.addArguments("--allow-insecure-localhost");

edited Jan 04 '18 at 01:21

dferenc

7,918
12
41
49

answered Jan 03 '18 at 23:37

singidunumx

101
1
5

One more clarification, the ui server must run on the localhost where the selenium test runs, i.e. https://localhost:443 – singidunumx Jan 04 '18 at 17:07

Gustavo Amaro · Answer 3 · 2018-09-22T02:13:49.667

6

Try this:

final ChromeOptions options = new ChromeOptions();

options.addArguments("--headless");
options.addArguments("--window-size=1280,800");

WebDriver driver = new ChromeDriver(options);

edited Sep 22 '18 at 02:13

answered Sep 22 '18 at 01:15

Gustavo Amaro

131
1
4
7

@Gustave, do you have a possible explanation as you why setting window-size changes the way test run happens? I have also noticed that when starting headless browser with chromeoption --start-maximized causes failure in headless mode. I am not sure what is the explanation for the same though. – aseema31 Jul 19 '19 at 21:34

score 2 · Answer 4 · answered Jun 24 '21 at 14:23

2

Adding the user-agent did the job for me:

--user-agent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36

answered Jun 24 '21 at 14:23

joanfihu

336
2
9

score 1 · Answer 5 · answered Mar 02 '23 at 19:17

1

options.add_argument("--window-size=1440, 900")

after trying out what others recommended, this I taken from "Slyme" solved my issue. mine is a java framework.

answered Mar 02 '23 at 19:17

WalkinInDark

11
1

1

As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Akzy Mar 03 '23 at 11:19

score 0 · Answer 6 · answered Nov 02 '17 at 05:51

Your chromedriver/selenium combination looks perfect. Seems to me a puely synchronization issue. We need to induce some wait to syck up as follows:

driver.get(url);
WebDriverWait wait = new WebDriverWait(driver, 10);
WebElement usernameTextfield = wait.until(ExpectedConditions.elementToBeClickable(driver.findElement(By.cssSelector(".input.username"))));
usernameTextfield.sendKeys("user_name");

Slyme · Answer 7 · 2020-05-16T20:55:36.443

For anyone who may stumble here with this error, but is utilizing python.

I was having issues with a script working only when headless was not utilized.

Initially, I had my options looking like this:

options = Options()
options.headless = True

After finding this thread, I altered my options to the following:

options = Options()
options.add_argument("--headless");
options.add_argument("--window-size=1440, 900")

It appears that headless windows are formatted differently when selenium navigates the page. Go figure. This solved all of my issues.

score 0 · Answer 8 · answered May 16 '20 at 22:46

0

I had the exact same problem.

You need to add to your options your computer's user agent, To search your user agent just type in google: ״my user agent״

Then add it to the options: options.add_argument("your-user-agent")

answered May 16 '20 at 22:46

Eilonlif

346
2
7

Getting this with use agent added: selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: crashed. – Sergo055 Jul 06 '23 at 13:25

selenium tests fail against headless chrome

8 Answers8

Linked