1

I want to retrieve the company from a website. If without while loop, the code can run properly. But with while loop, it gives me this error: Exception has occurred: StaleElementReferenceException Message: stale element reference: stale element not found when executed. And I can really find elements through the xpath.

My Code:

dropdown = driver.find_elements(by='xpath', value = '//div[@class="main-block flexible-submenu"]/section[@class="container border-bottom border-dark"]/div[@class=" tab-content"]/div/div/div/div[@class="dataTables_length"]/label/select/option')
i=0
while(i<len(dropdown)):
    if(dropdown[i].text=="All"):
        dropdown[i].click()
        
        lists = []
        stockLists = driver.find_elements(by='xpath', value = '//div[@class="main-block flexible-submenu"]/section[@class="container border-bottom border-dark"]/div[@class=" tab-content"]/div/div[@class]/div[@id="DataTables_Table_0_wrapper"]/div[@class="dataTables_scroll"]/div[@class="dataTables_scrollBody"]/table/tbody/tr/td[@class=" text-left position-relative"]')
        for stockList in stockLists:
            list = stockList.find_element(by="xpath", value = './a[@class="company-announcement-link"]').text
            lists.append(list)
        my_dict = {'stock': lists}
        df_headlines=pd.DataFrame(my_dict)
        df_headlines.to_csv('headline.csv')
        
    i=i+1

driver.quit()

Error on the line: list = stockList.find_element(by="xpath", value = './a[@class="company-announcement-link"]').text

Error Trace: Exception has occurred: StaleElementReferenceException Message: stale element reference: stale element not found Backtrace: GetHandleVerifier [0x00A88893+48451] (No symbol) [0x00A1B8A1] (No symbol) [0x00925058]

Ong Chan Meng
  • 11
  • 1

1 Answers1

0

This usually what happens when you iterate over elements in a loop. At some point, when you scroll down or op, or any other logic on the page - the element or elements you are running on in the loop no longer exist. You can fix this in 2 ways:

  1. Add try and except to your code. When you enter the except block - try to re-find you element.

  2. You are using a very long Xpath / CSS expression. That is a big no no. Always try to shorten your expressions. You can find and element by text using Xpath like this:

     text_xpath = '//*[contains(text(),'TextToFind')]'
 element = driver.find_element(by="xpath",
                               value = text_xpath)
 element.click()
Tal Angel
  • 1,301
  • 3
  • 29
  • 63