Selenium Webdriver Python is not looping

Question

I'm trying to use Selenium to click on multiple links one each a time to download multiple CSV files, the problem here is the selenium makes the donwload of about few csv files but in the middle of the loop it stops working, crashes the Browser accusin that don't have internet and close the driver. I already put the chromedriver.exe in the same folder and put the path but it still not working.

from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
import time
import os
import re

new_directory = r"Z:\BI_Database_teste"
for document in os.listdir(new_directory):
    os.remove(os.path.join(new_directory, document))

url = 'myPersonalURL'

chrome_options = Options()
chrome_options.add_argument('--headless')
browser = webdriver.Chrome('chromedriver.exe', options=chrome_options)
params = {'behavior': 'allow', 'downloadPath': new_directory}
browser.execute_cdp_cmd('Page.setDownloadBehavior', params)
browser.get(url)

time.sleep(2)


input_email = browser.find_element(By.ID, 'email')
input_email.send_keys('myEmail')
input_password = browser.find_element(By.ID, 'password')
input_password.send_keys('myPassword')
input_password.submit()

input_question = browser.find_element(By.XPATH, '/html/body/div[2]/div[2]/form/table/tbody/tr[3]/td/table/tbody/tr[2]/td[2]').text
answer_field = browser.find_element(By.XPATH, '/html/body/div[2]/div[2]/form/table/tbody/tr[3]/td/table/tbody/tr[3]/td[2]/input')

if input_question == 'question':
    answer_field.send_keys('answer')
elif input_question == 'question':
    answer_field.send_keys('answer')
else:
    answer_field.send_keys('answer')

time.sleep(2)

answer_field.submit()
time.sleep(4)


links = browser.find_elements(By.LINK_TEXT, 'Export (CSV)')
links_len = len(links)
print(str(links_len) + ' BI Databases to Download')
list_count = 0

for link in range(list_count, links_len):
    time.sleep(2)
    links[list_count].click()
    list_count = list_count + 1
    print(str(list_count) + ' BI Databases downloaded')

browser.quit()


for file in os.listdir(new_directory):
    if file.startswith("PBI"):
        try:
            os.rename(os.path.join(new_directory, file), os.path.join(new_directory, re.sub('[0-9]', '', file)))
        except:
            pass

print('BI Databases Download Successfully!')```


Could someone help me to find out why the webdriver stops working in the middle of the loop?

score 0 · Answer 1 · answered Dec 21 '22 at 23:06

If you think that for some reason the driver isn't loaded correctly then you can download the webdriver on runtime from your code, this might help,

from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities

chrome = webdriver.Chrome(ChromeDriverManager().install(), options=options)

without much further context, the question cannot be answered or reproduced.

Selenium Webdriver Python is not looping

1 Answers1