2

I was trying to get into introductory web scraping with selenium in Python, but I keep getting this mysterious error when I start up a basic Chrome instance:

LookupError: unknown encoding: idna

when using the following code:

from selenium import webdriver

url = 'http://www.webscrapingfordatascience.com/complexjavascript/'

driver = webdriver.Chrome()
driver.get(url)

I installed Chrome's necessary webdriver with brew cask install chromedriver.

I tried searching around the web for potential solutions, but it does not seem anyone else has asked this in relation to running selenium with Python, and this bug is also rather vague.

Edit

The question here's answer of doing an import encodings.idna gives me the new error of Module not found in Python.

In addition, my system is a mbp with Mac OS 10.11, Python is 3.7.2 (Clang 8.0.0), and pip is =19.0.3.

echo $PATH gives me the following output:

/Users/Michael/miniconda3/bin:/Users/Michael/intelpython3/bin:/Users/Michael/miniconda3/bin:/Users/Michael/miniconda3/bin:/opt/local/bin:/opt/local/sbin:/usr/local/sbin:/Users/Michael/anaconda3/lib/python3.6/site-packages:/Library/Frameworks/Python.framework/Versions/3.7/bin:/Users/Michael/anaconda3/bin:/Users/Michael/anaconda3/bin:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/MacGPG2/bin:/Library/TeX/texbin’:/Users/Michael/intelpython3/bin
halfer
  • 19,824
  • 17
  • 99
  • 186
Coolio2654
  • 1,589
  • 3
  • 21
  • 46
  • Possible duplicate of [Unknown encoding: idna in Python Requests](https://stackoverflow.com/questions/9144724/unknown-encoding-idna-in-python-requests) – Edeki Okoh Feb 20 '19 at 21:16

1 Answers1

1

This error message...

LookupError: unknown encoding: idna

...implies that there was an encoding / decoding error between idna and utf-8.

This error comes from _get_idna_encoded_host(host) method of models.py which is defined as follows:

@staticmethod
def _get_idna_encoded_host(host):
    import idna

    try:
        host = idna.encode(host, uts46=True).decode('utf-8')
    except idna.IDNAError:
        raise UnicodeError
    return host

A bit of your system details in terms of architecture and os would have helped us to debug your issue in a better way. However:

Solution

The solution is to add the following import:

import encodings.idna

Note: Ensure that pip is on the PATH and is 9.0.1 or better.

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
  • I added details about my system above. So, I do `echo $PATH` (output in edit), and I get the following output, indicative of the many Python installations I've tried over the years; however, currently I only have one Python installation, as indicated by `which python` giving `/usr/bin/python`. Since the import above still fails for me, how am I supposed to check `pip` is on the path above? – Coolio2654 Feb 21 '19 at 08:16
  • @Coolio2654 I am afraid as I won't be able to guide you properly on `pip` installation path. Possibly a quick google search will provide you some valuable pointers. – undetected Selenium Feb 21 '19 at 16:20