Python/Selenium Stale Element Reference in For Loop

Question

I'm having trouble looping through a list of company ID's and using them in a search bar. My code works just fine when the text file contains just one ID, but when I add a second ID to the list it won't even perform the last click for the first ID, let alone search for the second one, and then gives me a stale element reference exception. This is driving me insane so any help would be amazing. I'm not very experienced at all so bear with me if I don't understand a request for more information or your solution.

Code:

company_list = open('Company_List.txt')
for line in company_list:
  company_id = driver.find_element_by_xpath('//*[@id="SearchTopBar"]')
  company_id.send_keys(line)
  company_id.send_keys(Keys.ENTER)
  driver.implicitly_wait(10)
  driver.find_element_by_xpath('//*[@id="CompanyHeaderInfo_TearSheetReport_ReportImage"]/div/img').click()
  driver.implicitly_wait(10)

Traceback:

File "<ipython-input-5-3766dfd38c2f>", line 1, in <module>
runfile('C:/Users/kdixon/Desktop/Business_Intelligence/PROJECTS/Scripts/Pull Tearsheets.py', wdir='C:/Users/kdixon/Desktop/Business_Intelligence/PROJECTS/Scripts')

File "C:\Users\kdixon\AppData\Local\Continuum\anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 705, in runfile
execfile(filename, namespace)

File "C:\Users\kdixon\AppData\Local\Continuum\anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 102, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)

File "C:/Users/kdixon/Desktop/Business_Intelligence/PROJECTS/Scripts/Pull Tearsheets.py", line 42, in <module>
company_id.send_keys(Keys.ENTER)

File "C:\Users\kdixon\AppData\Local\Continuum\anaconda3\lib\site-packages\selenium\webdriver\remote\webelement.py", line 479, in send_keys
'value': keys_to_typing(value)})

File "C:\Users\kdixon\AppData\Local\Continuum\anaconda3\lib\site-packages\selenium\webdriver\remote\webelement.py", line 628, in _execute
return self._parent.execute(command, params)

File "C:\Users\kdixon\AppData\Local\Continuum\anaconda3\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 314, in execute
self.error_handler.check_response(response)

File "C:\Users\kdixon\AppData\Local\Continuum\anaconda3\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)

StaleElementReferenceException: stale element reference: element is not attached to the page document

(Session info: chrome=67.0.3396.99)
(Driver info: chromedriver=2.40.565498 (ea082db3280dd6843ebfb08a625e3eb905c4f5ab),platform=Windows NT 10.0.16299 x86_64)

score 1 · Accepted Answer · answered Jul 07 '18 at 09:29

According to the traceback, particularly the fact that the exception is thrown from the line company_id.send_keys(Keys.ENTER) and the type of the exception, it appears that the search box (with id SearchTopBar) is replaced with another element when you start typing.

I guess that if you'll send the Enter press together with the value, it will solve the problem. I.e.:

company_id.send_keys(line + Keys.ENTER)

In case it doesn't, open the Developer Tools in the browser and see which element replaces the SearchTopBar element when you start typing. In the code, after sending the line keystrokes to the company_id element, search for the element that you found to replaces it, and send the Enter keypress to that element instead of to the company_id.

YES! That was it! Thank you! – Kyle Dixon Jul 09 '18 at 15:17 — Kyle Dixon, Jul 09 '18 at 15:17

Andrei Suvorkov · Answer 2 · 2018-07-05T20:37:27.417

0

Try this:

from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait

company_id = WebDriverWait(driver, 10).until(
    EC.presence_of_element_located((By.XPATH, "//*[@id='SearchTopBar']"))
)

it will wait until element will be present in DOM

Full code:

from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait

company_list = open('Company_List.txt')
for line in company_list:
  company_id = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, "//*[@id='SearchTopBar']")))
  company_id.send_keys(line)
  company_id.send_keys(Keys.ENTER)
  driver.implicitly_wait(10)
  driver.find_element_by_xpath('//*[@id="CompanyHeaderInfo_TearSheetReport_ReportImage"]/div/img').click()
  driver.implicitly_wait(10)

edited Jul 05 '18 at 20:37

answered Jul 05 '18 at 20:20

Andrei Suvorkov

5,559
5
22
48

I've clicked through a lot of pages and inspected the search bar and it always has the same xpath, is that the wrong way to check for a new xpath or does that mean it is something else? – Kyle Dixon Jul 05 '18 at 20:27
When you find this element via xPath in browser dev tools on the second iteration, do you getting this element still highlighted? – Andrei Suvorkov Jul 05 '18 at 20:51
Yes it's still highlighted. When highlighted it says: input#SearchTopBar.cSearchBoxDisabled It also says that on the first iteration as well. – Kyle Dixon Jul 05 '18 at 20:55
Try to debug your code and check if on the second iteration your `line` is not `None` – Andrei Suvorkov Jul 06 '18 at 03:42

Python/Selenium Stale Element Reference in For Loop

2 Answers2