4

I am trying to make a http2 request through pycurl. I have installed the nghttp2 c library that libcurl needs and I am able to do an http2 request with curl.

I tried something like this with pycurl:

import pycurl
import cStringIO


if __name__ == '__main__':
    c = pycurl.Curl()

    buf = cStringIO.StringIO()

    c = pycurl.Curl()
    c.setopt(c.URL, 'https://nghttp2.org')
    c.setopt(pycurl.HTTP_VERSION, pycurl.VERSION_HTTP2)
    c.setopt(c.WRITEFUNCTION, buf.write)
    c.setopt(pycurl.VERBOSE, 1)
    c.perform()

    buf.close()

And the output that I get is:

* Rebuilt URL to: https://nghttp2.org/
*   Trying 106.186.112.116...
* Connected to nghttp2.org (106.186.112.116) port 443 (#0)
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
 CApath: none
* NPN, negotiated HTTP1.1
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* Server certificate:
*    subject: CN=nghttp2.org
*    start date: Jan 21 22:58:00 2017 GMT
*    expire date: Apr 21 22:58:00 2017 GMT
*    subjectAltName: nghttp2.org matched
*    issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
*    SSL certificate verify ok.
> GET / HTTP/1.1
Host: nghttp2.org
User-Agent: PycURL/7.43.0 libcurl/7.46.0 OpenSSL/1.0.1f zlib/1.2.8          libidn/1.28 nghttp2/1.21.0-DEV librtmp/2.3
Accept: */*
Connection: Upgrade, HTTP2-Settings
Upgrade: h2c
HTTP2-Settings:

< HTTP/1.1 200 OK
< Date: Tue, 14 Mar 2017 13:33:48 GMT
< Content-Type: text/html
< Last-Modified: Sun, 26 Feb 2017 12:19:00 GMT
< Etag: "58b2c7b4-19e1"
< Accept-Ranges: bytes
< Content-Length: 6625
< X-Backend-Header-Rtt: 0.001418
< Strict-Transport-Security: max-age=31536000
< Server: nghttpx
< Via: 2 nghttpx
< x-frame-options: SAMEORIGIN
< x-xss-protection: 1; mode=block
< x-content-type-options: nosniff
<
* Connection #0 to host nghttp2.org left intact

What am I doing wrong in this request?

Edit: As per the comment I made sure that pycurl uses OpenSSL/1.0.2k. Now the User-Agent header looks like this: User-Agent: PycURL/7.43.0 libcurl/7.46.0 OpenSSL/1.0.2k zlib/1.2.11 libidn/1.28 nghttp2/1.21.0-DEV librtmp/2.3 but I get the same result as above.

Donovan
  • 816
  • 1
  • 10
  • 16

2 Answers2

4

It seems that the pycurl script was using the wrong http version constant. The correct one is c.setopt(pycurl.HTTP_VERSION, pycurl.CURL_HTTP_VERSION_2_0).

Donovan
  • 816
  • 1
  • 10
  • 16
1

That libcurl version you're using here identifies that it was built with OpenSSL/1.0.1f. OpenSSL before 1.0.2 doesn't have ALPN support and that's the TLS extension required to negotiate HTTP/2 on most current HTTPS sites. (Some still support the older NPN extension but its getting rarer, especially since no browser uses NPN anymore.)

Presumably, the libcurl your curl command line tool uses that you say is successful, uses another install?

Finally, providing those custom headers you do will probably only risk breaking your operation rather than help it. When libcurl negotiates HTTP/2 with a site, it uses the necessary headers automatically.

Daniel Stenberg
  • 54,736
  • 17
  • 146
  • 222
  • Following your comment, I have installed OpenSSL 1.0.2a 19 Mar 2015. I tried my script again and I see that it uses the old version. How can I force pycurl to use the new version of OpenSSL? – Donovan Mar 14 '17 at 15:02
  • (You should go with 1.0.2**k** instead to avoid a lot of bugs.) You *probably* edit /etc/ld.so.conf, reload ldconfig or set `LD_LIBRARY_PATH`... – Daniel Stenberg Mar 14 '17 at 15:05
  • I managed to make pycurl use OpenSSL/1.0.2k but I still get the same result. I also removed the extra headers. – Donovan Mar 15 '17 at 09:26