0

Been following a web scraping tutorial on youtube and I'm getting this error due to the url. It's working perfectly in Spyder but all hell breaks loose when I try to run it in the Command Prompt.

I tried various answers from here (quote() that converts ":" to "%3A" and a few others like urlencode()) but it didn't work. Or may be I don't know how to make them work.

from bs4 import BeautifulSoup as soup
from urllib.request import urlopen as uReq
#from urllib.parse import quote

my_url = 'https://www.newegg.com/Video-Cards-Video-Devices/Category/ID-38?Tpk=graphics%20card'

# opening connection, grabbing information and closing connection
uClient = uReq(my_url)

This is the error message - 

Traceback (most recent call last):
  File "my_first_webscrape.py", line 8, in <module>
    uClient = uReq(my_url)
  File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 525, in open
    response = self._open(req, data)
  File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 548, in _open
    'unknown_open', req)
  File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 503, in _call_chain
    result = func(*args)
  File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 1387, in unknown_open
    raise URLError('unknown url type: %s' % type)
urllib.error.URLError: <urlopen error unknown url type: https>
eyllanesc
  • 235,170
  • 19
  • 170
  • 241
Faheem Rajwadkar
  • 1
  • Does this help? https://stackoverflow.com/a/52397402/8599834 – theberzi Apr 06 '19 at 08:06
  • I saw this but tried his first statement to see if that's an SSL issue. "I had the same error when I tried to open a url with https, but no errors with http". My code doesn't work with http either. – Faheem Rajwadkar Apr 06 '19 at 08:26
  • I ran your script and when I print `uClient` it gives the object ``. I don't get why you are having the error may be try upgrading dependencies. – Kartikey Singh Apr 06 '19 at 09:04
  • Does it work if you try to simply the URL to something like just `http://www.newegg.com`? – theberzi Apr 06 '19 at 09:39

0 Answers0