Webdriver error from Selenium on Google Colab

Question

I'm creating a scraper on google colab using Selenium but now doesnt work. Yes in the past but I dont know why now doesn't.

The code is:

#dependencies
!pip install selenium
!apt-get update 
!apt install chromium-chromedriver
!pip install fake-useragent
from selenium import webdriver
from fake_useragent import UserAgent

#options to chromedriver
ua = UserAgent()
userAgent = ua.random
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')
driver = webdriver.Chrome('chromedriver',chrome_options=chrome_options)
chrome_options.add_argument('--user-agent="'+userAgent+'"')

When a I run this code, colab show the next error:

"Message: Service chromedriver unexpectedly exited. Status code was: 1"

error screenshot

Do you know any solution? I've looked for an answer on other related topics but nothing works for me

Run Selenium on Google Colab and scrape organic results

Answer 1

This error message...

WebDriverException: Message: Service chromedriver unexpectedly exited. Status code was: 1

...was the result of an issue induced as the colab system was updated from v18.04 to ubuntu v20.04 LTS and with Ubuntu v20.04 LTS google-colaboratory no longer distributes chromium-browser outside of a snap package.

---

Solution

As a solution you can install a compatible version of Chromium without snap following the discussion How to install Chromium without snap?

---

Quick Fix

@mco-gh created a new notebook following @metrizable's guidance (details below) which is working perfect as of now: https://colab.research.google.com/drive/1cbEvuZOhkouYLda3RqiwtbM-o9hxGLyC

---

Chromium Browser from Debian buster repository

However to install chromium-browser from the Debian buster repository using the following code block published by @metrizable in the discussion Issues when trying to use Chromedriver in Colab

%%shell
# Ubuntu no longer distributes chromium-browser outside of snap
#
# Proposed solution: https://askubuntu.com/questions/1204571/how-to-install-chromium-without-snap

# Add debian buster
cat > /etc/apt/sources.list.d/debian.list <<'EOF'
deb [arch=amd64 signed-by=/usr/share/keyrings/debian-buster.gpg] http://deb.debian.org/debian buster main
deb [arch=amd64 signed-by=/usr/share/keyrings/debian-buster-updates.gpg] http://deb.debian.org/debian buster-updates main
deb [arch=amd64 signed-by=/usr/share/keyrings/debian-security-buster.gpg] http://deb.debian.org/debian-security buster/updates main
EOF

# Add keys
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys DCC9EFBF77E11517
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 648ACFD622F3D138
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 112695A0E562B32A

apt-key export 77E11517 | gpg --dearmour -o /usr/share/keyrings/debian-buster.gpg
apt-key export 22F3D138 | gpg --dearmour -o /usr/share/keyrings/debian-buster-updates.gpg
apt-key export E562B32A | gpg --dearmour -o /usr/share/keyrings/debian-security-buster.gpg

# Prefer debian repo for chromium* packages only
# Note the double-blank lines between entries
cat > /etc/apt/preferences.d/chromium.pref << 'EOF'
Package: *
Pin: release a=eoan
Pin-Priority: 500


Package: *
Pin: origin "deb.debian.org"
Pin-Priority: 300


Package: chromium*
Pin: origin "deb.debian.org"
Pin-Priority: 700
EOF

# Install chromium and chromium-driver
apt-get update
apt-get install chromium chromium-driver