Python socket HTTP 1.1 CONNECT request without a valid response

Question

Well, I just want to make the following simple program that tries to create an https tunel with www.google.com at port 443. I first tried the following code:

import socket

def main():
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.connect(("www.google.com", 80))
    request = "CONNECT www.google.com:443 HTTP/1.1\n\n"
    s.send(request.encode())
    print(s.recv(4096).decode())

main()

The result of that was the following:

HTTP/1.1 405 Method Not Allowed

Content-Type: text/html; charset=UTF-8

Referrer-Policy: no-referrer

Content-Length: 1592

Date: Wed, 16 Aug 2017 07:56:14 GMT

Connection: close



<!DOCTYPE html>
<html lang=en>
  <meta charset=utf-8>
  <meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">
  <title>Error 405 (Method Not Allowed)!!1</title>
  <style>
    *{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}
  </style>
  <a href=//www.google.com/><span id=logo aria-label=Google></span></a>
  <p><b>405.</b> <ins>That’s an error.</ins>
  <p>The request method <code>CONNECT</code> is inappropriate for the URL <code>/</code>.  <ins>That’s all we know.</ins>

That means that the server does not allow this request to be executed. So I thought that the problem was the port number. So I changed it to 443(which is the port for https connection). The code is that:

def main():
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.connect(("www.google.com", 443))
    request = "CONNECT www.google.com:443 HTTP/1.1\n\n"
    s.send(request.encode())
    print(s.recv(4096).decode())

main()

But it does not print out a valid respnse as it should have done. It gives me an empty response. The question to that is: "Why is that happening? How can I make it work properly?" Note: I don't want to use built-in urllib or urllib2 libraries. I want to do that with sockets.

Change request to this: ` request = "GET www.google.com:443 HTTP/1.1\r\n\r\n"` — RaminNietzsche, Aug 16 '17 at 08:19

bakatrouble · Answer 1 · 2017-08-16T08:59:28.600

6

HTTP

In your original connection to port 80 you are just using wrong Host:

import socket


def main():
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.connect(('google.com', 80))
    request = b'CONNECT google.com HTTP/1.1\n\n'
    s.send(request)
    print(s.recv(4096).decode())

main()

Response:

HTTP/1.0 200 Connection established

Or use GET method right away:

request = b'GET http://google.com HTTP/1.1\n\n'

Response is the same as to HTTPS request, google.com host doesn't work for some reason.

HTTPS

You should wrap your socket in ssl tunnel (not sure if correct term) in order to connect using HTTPS, and GET method is ready to use right after connection:

import socket
import ssl


def main():
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s = ssl.wrap_socket(s)
    s.connect(('google.com', 443))
    request = b'GET google.com HTTP/1.1\n\n'
    s.send(request)
    print(s.recv(4096).decode())

main()

Response:

HTTP/1.1 302 Found
Cache-Control: private
Content-Type: text/html; charset=UTF-8
Referrer-Policy: no-referrer
Location: https://www.google.ru/?gfe_rd=cr&ei=WwCUWc66L6qB3APs7ZPABA
Content-Length: 259
Date: Wed, 16 Aug 2017 08:20:43 GMT
Alt-Svc: quic=":443"; ma=2592000; v="39,38,37,35"

<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>302 Moved</TITLE></HEAD><BODY>
<H1>302 Moved</H1>
The document has moved
<A HREF="https://www.google.ru/?gfe_rd=cr&amp;ei=WwCUWc66L6qB3APs7ZPABA">here</A>.
</BODY></HTML>

edited Aug 16 '17 at 08:59

answered Aug 16 '17 at 08:21

bakatrouble

1,746
13
19

First of all thank you for your answer. Secondly, is it sure that the code you wrote without wraping is working? Cause I interpreted that but nothing changed – Anastassis Kapetanakis Aug 16 '17 at 12:54
Yes, I'm getting "200 connection established" response. I'm using Python 3.6, maybe that's the issue? – bakatrouble Aug 16 '17 at 12:58
I use Python 6.2 and I copy-paste the first http code. Unfortunately it doesn't do anything! It returns a 405 Status Code[Method not allowed] – Anastassis Kapetanakis Aug 16 '17 at 13:46
6.2? Did you try sending get request? – bakatrouble Aug 16 '17 at 13:49
The GET request works properly. The problem occurs in CONNECT request – Anastassis Kapetanakis Aug 16 '17 at 13:55
To help you understand what I am talking about, I am trying to make a simple proxy server in which I connect my browser. – Anastassis Kapetanakis Aug 16 '17 at 14:01

Python socket HTTP 1.1 CONNECT request without a valid response

1 Answers1

HTTP

HTTPS