sock.recv() is taking too much time to execute in the python code

Question

Following is the code which listens on a port for HTTP requests and sends the request packet to the server running on port 80, gets the response and sends the data back to the client. Now, everything is executing fine but the following line of code :

data = req_soc.recv(1024)

is taking too much time to execute and I have observed that, it takes long time to execute when it is going to/has received the last packet. I have also tried the same code using select.select() but the results are the same. Since I want to handle the data (raw) that is coming from the client and the actual HTTP server, I have no other choice than using sockets.

import socket
import thread

def handle_client(client):
    data = client.recv(512)
    request = ''
    request += data
    print data

print '-'*20
spl = data.split("\r\n")

print spl[0]
print spl[1]

if len(request):
    req_soc = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    req_soc.connect(('localhost', 80))
    req_soc.send(request)

    response = ''
    data = req_soc.recv(1024)
    while data:
        response += data
        print 1
        data = req_soc.recv(1024)
    req_soc.close()
    print response
    if len(response):
        client.send(response)
client.close()

server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(('localhost', 4422))
server.listen(5)

print("Server is running...\n")

MSGLEN = 1024

while 1:

    client, address = server.accept()
    thread.start_new_thread(handle_client, (client, ))

score 0 · Accepted Answer · answered Mar 23 '12 at 15:19

0

Clients can do multiple commands (eg: GET) within one connection. You cannot wait for the client to send all the commands because based on what you return it could request more (eg: images of a web page). You have to parse the parts (commands) of request, find the boundary, forward that request to the server and write back the answer to the client. All this in a way that doesn't block on reading the client.

I'm not sure what's the best way to do this in python, but if you spend 5 minutes of googling you'll find a perfect HTTP proxy library.

answered Mar 23 '12 at 15:19

Karoly Horvath

94,607
11
117
176

Actually this is my project, so I cannot use any library. I will have to do that using sockets. Also when I receive request from client I am getting a single HTTP GET request at one time. – vedarthk Mar 23 '12 at 15:26
your clients have to do a write shutdown on the socket after sending the whole message otherwise your recv will continue to block... also note that I didn't say that you should *use* the library, you can just check the *source* – Karoly Horvath Mar 23 '12 at 15:47
OK. Sorry for the misunderstanding. – vedarthk Mar 23 '12 at 16:40
Done ! Thank you. I changed my algorithm and it is working smoothly. – vedarthk Mar 23 '12 at 19:06

sock.recv() is taking too much time to execute in the python code

1 Answers1