0

I am testing an ECM system using Selenium WebDriver for Python and, for the last couple of weeks, all my tests that previously worked are now failing in Chrome and Edge browsers.

Essentially, I can log in, but once I get to any page in the system, Selenium fails to find any element on the page. In fact, it can see that there are elements in the DOM, but cannot access them.

I have the following code:

elements = driver.find_elements(By.XPATH, "//*")
print(len(elements))
for element in elements:
    print(type(element))

Running this on the login page (before logging in) or on several other websites, I get something like this:

68
<class 'selenium.webdriver.remote.webelement.WebElement'>
<class 'selenium.webdriver.remote.webelement.WebElement'>
<class 'selenium.webdriver.remote.webelement.WebElement'>
<class 'selenium.webdriver.remote.webelement.WebElement'>
...

However, running it on the landing page of my system, I get this:

987
<class 'NoneType'>
<class 'NoneType'>
<class 'NoneType'>
<class 'NoneType'>
...

And, finally, running it on the exact same page, but in Firefox instead, I get this:

15
<class 'selenium.webdriver.remote.webelement.WebElement'>
<class 'selenium.webdriver.remote.webelement.WebElement'>
<class 'selenium.webdriver.remote.webelement.WebElement'>
<class 'selenium.webdriver.remote.webelement.WebElement'>
...

Which is the expected result when calling findElements(). What could possibly have happened in Chrome and Edge in the last few weeks that caused this?

Ciprian Prohozescu
  • 1
  • 1

1 Answers1

0

To start with, this locator strategy:

elements = driver.find_elements(By.XPATH, "//*")

isn't an effective locator strategy as it tries to identify all the elements on the webpage affected by the implicit wait time in force at the time of execution.

These usecases

  • Running the script on the login page (before logging in): Only 68 elements renders before the implicit wait time expires where as ideally the locator should match almost all the elements on the page i.e. more then tens of thousands.

  • Running the script on the landing page of my system: 987 elements renders before the implicit wait time expires but due to user settings and/or associated JavaScript or AJAX Calls none of the elements have completely rendered. Hence you observe:

    987
<class 'NoneType'>
<class 'NoneType'>
<class 'NoneType'>
<class 'NoneType'>

Finally it's worth to mention that, different browers uses different rendering mechanism, so the count of elements in generic cases may differ when using browsers like or

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
  • I'm not sure I follow. I only used that locator strategy for debugging purposes. Using a concrete XPATH or ID throws a NoSuchElement or Timeout exception, because, like you said, none of those 987 elements are fully rendered. The problem is, no matter how long I wait, they never get fully rendered. After some digging, I found a JS script that, once removed, solves this issue. But I cannot remove that script in the live system and it is too long and obscure to parse. What could possibly cause this behaviour? – Ciprian Prohozescu Jun 05 '23 at 22:40
  • @CiprianProhozescu The basic flaw is `elements = driver.find_elements(By.XPATH, "//*")` isn't an effective locator strategy. You won't require all the elements on a webpage for any such usecase. – undetected Selenium Jun 05 '23 at 22:48