3

I am working in a web scraping project with python 3.7. Done the code in python with selenium and chromediver.exe in windows, it's working fine. we are adding the script in aws lambda. the issue is we need to specify the chrome driver of Linux.

I followed the steps in https://github.com/yai333/Selenium-UI-testing-with-AWS-Lambda-Layers.

i am not using any serverless yml sript(i don't know the same). doing the following

we have a Linux machine. create a virtual python environment and add selenium module(as described in awshttps://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.html#python-package-venv) download chromdriver and headless into a folder(size is large, so upload to S3) add both (chrome driver and python lib) as layers. paste the handler.py (in https://github.com/yai333/Selenium-UI-testing-with-AWS-Lambda-Layers) to a lambdahandler file. create a sample test, and click on test. shows error: error message 'chromedriver' executable needs to be in path

can I upload chrome driver in S3 and show the path.

newuser
  • 61
  • 1
  • 10

1 Answers1

1

Having just fought this exact issue for a few hours, I think I can help.

Within your Lambda Layer, you need to include the chromedriver binary under /bin. Os it will look something like:

layerZip/
|- bin/
   |- chromedriver

Within your lambda function's infrastructure, it will exist at /opt/bin/chromedriver. As such, we need to point our Python towards that as the executable. To get it to work, I had to add the following:

from selenium import webdriver

chrome_options = webdriver.ChromeOptions()
chrome_options.binary_location = "/opt/bin/chromedriver"
driver = webdriver.Chrome(executable_path="opt/bin/chromedriver", options=chrome_options)
DevOpsIsNotForMe
  • 29
  • 8
  • 3
    I should add, I'm not convinced you need the chrome_options.binary_location part. I'm also not certain that chromedriver needs to appear under /bin, it might be okay to just appear in layer.zip at top level. In this case, I think it would appear at /opt/chromedriver. It's been a long day and my username is really holding its own right now. – DevOpsIsNotForMe Nov 11 '20 at 17:09