HTTPS connection using PEM Certificate

Question

I'm trying to POST HTTPS requests using a PEM certificate like following:

import httplib  
CERT_FILE = '/path/certif.pem'
conn = httplib.HTTPSConnection('10.10.10.10','443', cert_file =CERT_FILE)   
conn.request("POST", "/") 
response = conn.getresponse()       
print response.status, response.reason
conn.close()

I have the following error:

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.6/httplib.py", line 914, in request
self._send_request(method, url, body, headers)
File "/usr/lib/python2.6/httplib.py", line 951, in _send_request
self.endheaders()
File "/usr/lib/python2.6/httplib.py", line 908, in endheaders
self._send_output()
File "/usr/lib/python2.6/httplib.py", line 780, in _send_output
self.send(msg)
File "/usr/lib/python2.6/httplib.py", line 739, in send
self.connect()
File "/usr/lib/python2.6/httplib.py", line 1116, in connect
self.sock = ssl.wrap_socket(sock, self.key_file, self.cert_file)
File "/usr/lib/python2.6/ssl.py", line 338, in wrap_socket
suppress_ragged_eofs=suppress_ragged_eofs)
File "/usr/lib/python2.6/ssl.py", line 118, in __init__
cert_reqs, ssl_version, ca_certs)
ssl.SSLError: [Errno 336265225] _ssl.c:339: error:140B0009:SSL       
routines:**SSL_CTX_use_PrivateKey_file**:PEM lib

When I remove the cert_file from httplib, I've the following response:

200 ok

When I add the Authentication header (like advised by MattH) with empty post payload, it works also.

However, when I put the good request with the Path, the Body and the Header, like following (I simplified them...)

body = '<S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/">blablabla</S:Envelope>'
URLprov = "/syncaxis2/services/XXXsyncService"
auth_header = 'Basic %s' %  (":".join(["xxx","xxxxx"]).encode('Base64').strip('\r\n'))
conn.request("POST",URLprov,body,{'Authenticate':auth_header})

I have 401 Unauthorized response !

As you can see, first, I'm asked to provide the PrivateKey ! why did I need the PrivateKey if I'm a client ? then, when I remove the PrivateKey and the certificate, and put the Path/Body/headers I have 401 Unauthorized error with the message WWW-Authenticate: Basic realm="SYNCNB Server Realm".

Could any one explain this issue? Is there another way to send HTTPS request using a certificate in Python?

score 9 · Answer 1 · edited May 23 '17 at 11:45

It sounds like you need something similar to an answer I have provided before to perform simple client certificate authentication. Here is the code for convenience modified slightly for your question:

import httplib
import urllib2

PEM_FILE = '/path/certif.pem' # Renamed from PEM_FILE to avoid confusion
CLIENT_CERT_FILE = '/path/clientcert.p12' # This is your client cert!

# HTTPS Client Auth solution for urllib2, inspired by
# http://bugs.python.org/issue3466
# and improved by David Norton of Three Pillar Software. In this
# implementation, we use properties passed in rather than static module
# fields.
class HTTPSClientAuthHandler(urllib2.HTTPSHandler):
    def __init__(self, key, cert):
        urllib2.HTTPSHandler.__init__(self)
        self.key = key
        self.cert = cert
    def https_open(self, req):
        #Rather than pass in a reference to a connection class, we pass in
        # a reference to a function which, for all intents and purposes,
        # will behave as a constructor
        return self.do_open(self.getConnection, req)
    def getConnection(self, host):
        return httplib.HTTPSConnection(host, key_file=self.key, cert_file=self.cert)


cert_handler = HTTPSClientAuthHandler(PEM_FILE, CLIENT_CERT_FILE)
opener = urllib2.build_opener(cert_handler)
urllib2.install_opener(opener)

f = urllib2.urlopen("https://10.10.10.10")
print f.code

Hi jathanism, in fact, I have only the client_cert_file.pem (a base64 file) because I'm the client which will connect to the server, and I want to use it in order to send HTTPS request to a server. Maybe I'm not using the good way to ask servers using SSL PEM certificate ? — psikais, May 05 '11 at 14:47
@jathanism your code is throwing off an error. TypeError: getConnection() got an unexpected keyword argument 'timeout' — code base 5000, Jun 12 '15 at 11:18
You can fix that by changing the line to `getConnection(self, host, timeout=300):` — relet, Jun 29 '17 at 08:29

MattH · Accepted Answer · 2011-05-06T13:13:39.123

2

See http://docs.python.org/library/httplib.html

httplib.HTTPSConnection does not do any verification of the server’s certificate.

The option to include your private certificate is when the server is doing certificate based authentication of clients. I.e. the server is checking the client has a certificate signed by a CA that it trusts and is allowed to access it's resources.

If you don't specify the cert optional argument, you should be able to connect to the HTTPS server, but not validate the server certificate.

Update

Following your comment that you've tried basic auth, it looks like the server still wants you to authenticate using basic auth. Either your credentials are invalid (have you independently verified them?) or your Authenticate header isn't formatted correctly. Modifying your example code to include a basic auth header and an empty post payload:

import httplib  
conn = httplib.HTTPSConnection('10.10.10.10','443')   
auth_header = 'Basic %s' % (":".join(["myusername","mypassword"]).encode('Base64').strip('\r\n'))
conn.request("POST", "/","",{'Authorization':auth_header}) 
response = conn.getresponse()       
print response.status, response.reason
conn.close()

edited May 06 '11 at 13:13

answered May 05 '11 at 11:03

MattH

37,273
11
82
84

Hi MattH, In fact, when I don't specify the cert optional argument, I have the response 401 Unauthorized... Could you please tell my how to do to verify the certificate I have in order to be accepted by the distant server ? Many thanks for your help – psikais May 05 '11 at 12:10
@psikais: The service requires some form of authentication. Check the "WWW-Authenticate" header in the server 401 response for more information about what authentication schemes it supports. – MattH May 05 '11 at 12:37
@ MattH, I've had the error message "WWW-Authenticate: Basic realm="SYNCNB Server Realm". Also, as I told you, I did not use my certificate anywhere... – psikais May 05 '11 at 13:02
The server requires that you authenticate using Basic authentication. This is a username and password encoded and included as a header in your http request. – MattH May 05 '11 at 13:08
I put the user and password within the header, but I still have the same error: >>> print response.status, response.reason, response.msg 401 Unauthorized Date: Thu, 05 May 2011 13:43:30 GMT Server: Apache-Coyote/1.1 WWW-Authenticate: Basic realm="SYNCNB Server Realm" Content-Type: text/html;charset=utf-8 Content-Length: 954 – psikais May 05 '11 at 13:46
In fact, I've updated my post above. could you please see again and advice ? the unauthorized error occurs only when I add the Path (URLprov) in the request...even if I add the Authentication information – psikais May 05 '11 at 16:34
@psikais: It seems that the credentials that you're using may not be authorised to access this SOAP resource. I suggest that you consult any documentation that came with this System that you are attempting to access. It should spell out authentication and authorisation requirements. Perhaps the username you are using needs particular permissions configured. – MattH May 05 '11 at 18:50
Hi MattH, in fact I've just replaced 'Authenticate':auth_header by 'Authorization':auth_header....and it worked. As you told me, the certificate is not used at all in my case. Thanks a lot for your great help! – psikais May 06 '11 at 12:15
@psikais: Oops sorry, clearly I fudged the header name while composing that little snippet. I've updated it. – MattH May 06 '11 at 13:15

score 1 · Answer 3 · edited May 23 '17 at 11:50

What you're doing is trying to connect to a Web service that requires authentication based on client certificate.

Are you sure you have a PEM file and not a PKCS#12 file? A PEM file looks like this (yes, I know I included a private key...this is just a dummy that I generated for this example):

Read this question.

Hi A,J Yes I'm sure, my certifacte is something like : -----BEGIN CERTIFICATE----- blablaba -----END CERTIFICATE----- — psikais, May 05 '11 at 14:15

HTTPS connection using PEM Certificate

3 Answers3

Related