3

I am experiencing a strange behaviour from selenium when running it on headless mode with Chrome webdriver. Up to now, I did not have this problem before to get text in headless mode, it always worked.

The reproduceble example is given bellow:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

options = webdriver.ChromeOptions()
#options.add_argument('--headless')
#options.add_argument('--no-sandbox')

driver = webdriver.Chrome(chrome_options=options)

driver.get("https://www.zoom.com.br/ar-condicionado/todos")

wait = WebDriverWait(driver, 10)

stores = wait.until(
    EC.presence_of_all_elements_located((By.XPATH,
                                        './/span[@class="storeCount-txt"]')))

print(stores[0].text)

When I ran this peace of code the output is:

> em 14 lojas

However, when I run it on headless mode (remove #s), the output is empty:

> ""

Any ideas on what is going on?

Everton Reis
  • 422
  • 4
  • 16

2 Answers2

1

I had the same issue when I was running my webcrawling script deployed in heroku running chrome in headless mode. I solved it by adding the following chrome option to my list of options

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--disable-gpu')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument("--start-maximized")
chrome_options.add_argument("--window-size=1920x1080") #I added this

Just as mentioned on the comment in your question, there could be two things that could make some elements not show up

  1. The resolution you are on does not display that element(or something like that) which I have solved by adding that option
  2. You are searching for the element when it is not yet loaded. I suggest waiting appropriately (which you have done with the stores variable) you could also use this instead

     try:
        # Wait until 'what you specified' is visible
        WebDriverWait(driver, 60) \
            .until(expected_conditions.visibility_of_element_located((By.XPATH, './/span[@class="storeCount-txt"]')))
     except Exception as exp:
        print("Exception occured", exp)
        driver.quit()
    

Hope this helps

0

Try this.This should work.

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.add_argument('--no-sandbox')
options.add_argument("--start-maximized")

driver = webdriver.Chrome(chrome_options=options,executable_path='D:/Java/TestChrome/lib/chromedriver.exe')

driver.get("https://www.zoom.com.br/ar-condicionado/todos")
wait = WebDriverWait(driver, 20)

stores = wait.until(EC.presence_of_all_elements_located((By.XPATH,'//span[@class="storeCount-txt"]')))
print("test : " + stores[0].get_attribute('innerHTML'))

Let me know if this work.

KunduK
  • 32,888
  • 5
  • 17
  • 41
  • 3
    I guess it would be better for OP to know *why* this should work – JaSON Feb 01 '19 at 22:23
  • Because java scripts call here.so if you want to get value from it.you should take the attribute value instead of text.I guess you got it what i mean. – KunduK Feb 01 '19 at 22:42