HTTPS request with Python standard library

Question

UPDATE: I managed to do a request with urllib2, but I'm still wondering what is happening here.

I would like to do a HTTPS request with Python.

This works fine with the requests module, but I don't want to use external dependencies, so I'd like to use the standard library.

httplib

When I follow this example I don't get a response. I get a timeout instead. I'm out of ideas as to what would cause this.

Code:

import requests
print requests.get('https://python.org')

from httplib import HTTPSConnection
conn = HTTPSConnection('www.python.org')
conn.request('GET', '/index.html')
print conn.getresponse()

Output:

<Response [200]>
Traceback (most recent call last):
  File "test.py", line 6, in <module>
    conn.request('GET', '/index.html')
  File "C:\Python27\lib\httplib.py", line 1069, in request
    self._send_request(method, url, body, headers)
  File "C:\Python27\lib\httplib.py", line 1109, in _send_request
    self.endheaders(body)
  File "C:\Python27\lib\httplib.py", line 1065, in endheaders
    self._send_output(message_body)
  File "C:\Python27\lib\httplib.py", line 892, in _send_output
    self.send(msg)
  File "C:\Python27\lib\httplib.py", line 854, in send
    self.connect()
  File "C:\Python27\lib\httplib.py", line 1282, in connect
    HTTPConnection.connect(self)
  File "C:\Python27\lib\httplib.py", line 831, in connect
    self.timeout, self.source_address)
  File "C:\Python27\lib\socket.py", line 575, in create_connection
    raise err
socket.error: [Errno 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

urllib

This fails for a different (but possibly related) reason. Code:

import urllib
print urllib.urlopen("https://python.org")

Output:

Traceback (most recent call last):
  File "test.py", line 10, in <module>
    print urllib.urlopen("https://python.org")
  File "C:\Python27\lib\urllib.py", line 87, in urlopen
    return opener.open(url)
  File "C:\Python27\lib\urllib.py", line 215, in open
    return getattr(self, name)(url)
  File "C:\Python27\lib\urllib.py", line 445, in open_https
    h.endheaders(data)
  File "C:\Python27\lib\httplib.py", line 1065, in endheaders
    self._send_output(message_body)
  File "C:\Python27\lib\httplib.py", line 892, in _send_output
    self.send(msg)
  File "C:\Python27\lib\httplib.py", line 854, in send
    self.connect()
  File "C:\Python27\lib\httplib.py", line 1290, in connect
    server_hostname=server_hostname)
  File "C:\Python27\lib\ssl.py", line 369, in wrap_socket
    _context=self)
  File "C:\Python27\lib\ssl.py", line 599, in __init__
    self.do_handshake()
  File "C:\Python27\lib\ssl.py", line 828, in do_handshake
    self._sslobj.do_handshake()
IOError: [Errno socket error] [SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:727)

What is requests doing that makes it succeed where both of these libraries fail?

Do you have to use Python 2? – chepner Mar 19 '20 at 11:48 — chepner, Mar 19 '20 at 11:48
Yes, and ideally it is also compatible with python 3 – ToonAlfrink Mar 19 '20 at 12:49 — ToonAlfrink, Mar 19 '20 at 12:49

score 1 · Answer 1 · answered Mar 19 '20 at 11:47

1

requests.get without timeout parameter mean no timeout at all. httplib.HTTPSConnection accept parameter timeout in Python 2.6 and newer according to httplib docs. If your problem was caused by timeout, setting high enough timeout should help. Please try replacing:

conn = HTTPSConnection('www.python.org')

with:

conn = HTTPSConnection('www.python.org', timeout=300)

which will give 300 seconds (5 minutes) for processing.

answered Mar 19 '20 at 11:47

Daweo

31,313
3
12
25

This doesn't work. It just waits 300 seconds and then it gives the same error. – ToonAlfrink Mar 19 '20 at 12:49

HTTPS request with Python standard library

1 Answers1