WebDriverException : "Process unexpectedly closed with status 255" - selenium/geckodriver/AWS lambda - Python

Question

I am currently working on a web scraping project on aws lambda (serverless)

techs :

python3

selenium 3.14

geckodriver-V0.29

firefox 80.0 (en mode headless)

Here is the code I implemented (knowing that I configured the paths for firefox and geckodriver beforehand) :

from selenium import webdriver
from selenium.webdriver.firefox.options import Options as FirefoxOptions
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary

options = FirefoxOptions()
options.add_argument("--headless")
binary = FirefoxBinary("/tmp/bin/firefox/firefox")

webdriver.Firefox(options=options,executable_path="/tmp/bin/geckodriver", firefox_binary=binary)

This is the error I get:

Message: Process unexpectedly closed with status 255
: WebDriverException
Traceback (most recent call last):
  File "/var/task/src/lambda_function.py", line 8, in lambda_handler
    driver = WebDriverWrapper()
  File "/var/task/src/webdriver_wrapper.py", line 116, in __init__
    self._driver = webdriver.Firefox(options=options,executable_path="/tmp/bin/geckodriver", firefox_binary=binary)
  File "/var/task/lib/selenium/webdriver/firefox/webdriver.py", line 174, in __init__
    keep_alive=True)
  File "/var/task/lib/selenium/webdriver/remote/webdriver.py", line 157, in __init__
    self.start_session(capabilities, browser_profile)
  File "/var/task/lib/selenium/webdriver/remote/webdriver.py", line 252, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/var/task/lib/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/var/task/lib/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: Process unexpectedly closed with status 255

I have searched the forums and articles but have not found a solution. Has anyone experienced this kind of problem and if possible how to correct it?

Matteo Pasini · Answer 1 · 2022-07-05T13:27:13.273

11

I've had the same problem these days, apparently it's due to the firefox executables being downloaded directly from the link inside a docker container, which doesn't have a set of libraries firefox needs to work. To solve the problem I therefore had to add the following line to the Dockerfile.

RUN apt-get update && apt-get install -y wget bzip2 libxtst6 libgtk-3-0 libx11-xcb-dev libdbus-glib-1-2 libxt6 libpci-dev && rm -rf /var/lib/apt/lists/*

To test which libraries were missing, I connected to the docker container and try to run firefox in the following way:

firefox -headless

edited Jul 05 '22 at 13:27

answered Mar 03 '22 at 10:50

Matteo Pasini

1,787
3
13
26

2

Thank you, that was really helpful to me! :) One suggestion though: It should be sufficient to use `libgtk-3-0` instead of `packagekit-gtk3-module`, as the latter will pull in in whole lot of additional dependencies. – ahuemmer Jul 05 '22 at 13:23
1

It works, but indeed only when adding these to the Dockerfile. I'm still unsure why it does not work when executing this in the container. I also removed the last bit `rm -rf /var/lib/apt/lists/*` as this does not seem necessary to achieve the result. – Pepijn Olivier Dec 21 '22 at 15:06

score 2 · Answer 2 · answered Apr 14 '21 at 10:55

2

It's an issue about Firefox. Try updating Firefox to the latest version. I think Firefox 80.0 is incompatible with geckodriver v0.29

answered Apr 14 '21 at 10:55

TheFakeWater

81
4

Jefferey Cave · Answer 3 · 2021-12-11T15:35:32.810

(late answer, left for the next poor soul trying to solve the problem)

You are running headless. Is this running in a VM or Container with no head? Did you install FF from tar?

For me this turned out to be that the instance I was running did not have a desktop installed, and therefore was missing some libraries Firefox depends on. I had installed Firefox from download and unpacked.

To prove this was the problem, I installed Firefox from repo, and then ran my unpacked instance. Seeing that work demonstrated that I was simply missing dependancy libraries.

I have still not figured out the minimal set of libraries to install from Repo.

EDIT: I have found gitpod's gecko container to be informative.

https://github.com/gitpod-io/workspace-images/blob/master/gecko/Dockerfile

WebDriverException : "Process unexpectedly closed with status 255" - selenium/geckodriver/AWS lambda - Python

3 Answers3