Fill out form and download zipfile using Selenium in Python

Question

EDITED: I incorporated the final lines suggested by Sushil. At the end, I am copying the output in my terminal. I still do not get the zipfile.

SOLVED: My error was due to an incompatibility between the driver and chrome versions. I fixed by following the instructions here: unknown error: call function result missing 'value' for Selenium Send Keys even after chromedriver upgrade

I am trying to use Selenium to fill out a form and download a zipfile. After extensively googling, I have written a Python code, but I am currently unable to download the file. A browser opens, but nothing is filled out.

I am very new at Python, so I am guessing I am missing something very trivial since the website I am trying to get info from is super simple.

This is what I have tried:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys

driver = webdriver.Chrome(executable_path='/home/miranda/webscrap_Python/chromedriver')
#driver.wait = WebDriverWait(driver,5)
driver.get("http://www.cis.es/cis/opencms/EN/formulario.jsp?dwld=/Microdatos/MD3288.zip")

Name = '//*[@id="Nombre"]'
LastName = '//*[@id="Apellidos"]'
University = '//*[@id="profesion"]'
email = '//*[@id="Email"]'

ob_req = '//*[@id="objeto1"]'
terms = '//*[@id="Terminos"]'

download = '//*[@id="mediomicrodatos"]/form/div[3]/input'

driver.find_element_by_xpath(Name).send_keys("Miranda")

driver.find_element_by_xpath(LastName).send_keys("MyLastName")

driver.find_element_by_xpath(University).send_keys("MySchool")

driver.find_element_by_xpath(email).send_keys("my_email@gmail.com")

#lines added by Sushil:

ob_req_element =  driver.find_element_by_xpath(ob_req) #Finds the ob_req element
driver.execute_script("arguments[0].click();", ob_req_element) #Scrolls down to the element and clicks on it

terms_element = driver.find_element_by_xpath(terms) #The same is repeated here
driver.execute_script("arguments[0].click();", terms_element)

driver.find_element_by_xpath(download).click() #Scrolling down is not needed for the download button as it would already be in view. Only when an element is not in view should we scroll down to it in order to click on it.

Output in my terminal:

Traceback (most recent call last):
  File "myCode.py", line 18, in <module>
    driver.find_element_by_xpath(Name).send_keys("Miranda")
  File "/home/miranda/anaconda3/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py", line 479, in send_keys
    'value': keys_to_typing(value)})
  File "/home/miranda/anaconda3/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py", line 633, in _execute
    return self._parent.execute(command, params)
  File "/home/miranda/anaconda3/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/home/miranda/anaconda3/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: call function result missing 'value'
  (Session info: chrome=86.0.4240.75)
  (Driver info: chromedriver=2.29.461571 (8a88bbe0775e2a23afda0ceaf2ef7ee74e822cc5),platform=Linux 5.4.0-45-generic x86_64)

Have you done any debugging? I would recommend reading https://ericlippert.com/2014/03/05/how-to-debug-small-programs/. — AMC, Oct 18 '20 at 02:26
Thank you. I have read very carefully the output in the terminal, but I found it very confusing. Usually, the terminal is very helpful, since it indicates the lines with problems. This time, the terminal is referencing lines that I do not even have on my code, so I am not sure how to address that. The terminal only mentions an error twice: self.error_handler.check_response(response); and elenium.common.exceptions.WebDriverException: Message: unknown error: call function result missing 'value'. I have been googling them, but so far I am still struggling. — Miranda, Oct 18 '20 at 02:40
You wrote in your post that _A browser opens, but nothing is filled out._ There is an error, too? — AMC, Oct 18 '20 at 02:40
@AMC, Sorry, I am not sure what you mean. I usually use R, and when I try to do this procedure in R, I can open the page and fill out the form. What I meant by that was that in Python I can only open the tab, but have no success in filling the form. — Miranda, Oct 18 '20 at 02:46
If you are getting error messages, update your question with them. — JeffC, Oct 18 '20 at 04:03

Sushil · Accepted Answer · 2020-10-18T05:31:42.250

For each and every element below the email element, you have to scroll down to click them. Here is the full code to do it:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys

driver = webdriver.Chrome()
#driver.wait = WebDriverWait(driver,5)
driver.get("http://www.cis.es/cis/opencms/EN/formulario.jsp?dwld=/Microdatos/MD3288.zip")

Name = '//*[@id="Nombre"]'
LastName = '//*[@id="Apellidos"]'
University = '//*[@id="profesion"]'
email = '//*[@id="Email"]'

ob_req = '//*[@id="objeto1"]'
terms = '//*[@id="Terminos"]'

download = '//*[@id="mediomicrodatos"]/form/div[3]/input'

driver.find_element_by_xpath(Name).send_keys("Miranda")

driver.find_element_by_xpath(LastName).send_keys("MyLastName")

driver.find_element_by_xpath(University).send_keys("MySchool")

driver.find_element_by_xpath(email).send_keys("my_email@gmail.com")

#All the lines below were added by me

ob_req_element =  driver.find_element_by_xpath(ob_req) #Finds the ob_req element
driver.execute_script("arguments[0].click();", ob_req_element) #Scrolls down to the element and clicks on it

terms_element = driver.find_element_by_xpath(terms) #The same is repeated here
driver.execute_script("arguments[0].click();", terms_element)

driver.find_element_by_xpath(download).click() #Scrolling down is not needed for the download button as it would already be in view. Only when an element is not in view should we scroll down to it in order to click on it.

Thank you! I have added your lines, but it seems that I still do not get the zipfile. I updated my question with the error message that appears in my terminal. — Miranda, Oct 18 '20 at 18:04

Fill out form and download zipfile using Selenium in Python

1 Answers1