python change max limit recv buffer in UDPServer

Question

I use UDPServer based on BaseServer to receive frag UDP packets .

But some packets are larger than 8192 bytes(@handle method, print len(data)), and I can't use them correctly.

My original code:

class MyUDPHandler(SocketServer.BaseRequestHandler):
    def handle(self):
        global arr_len_recv    
        data = self.request[0].strip()
        socket = self.request[1]
        s_recv_paylaod_len = len(data)
        print "s_paylaod",binascii.hexlify(data)

if __name__ == '__main__':    
    server = SocketServer.UDPServer((HOST, LISTEN_PORT), MyUDPHandler)
    tmp = threading.Thread(None, loop_send, None,(param,server) , {})
    tmp.daemon=True
    tmp.start()
    poll=0.1
    server.serve_forever(poll)

So I RTFM BaseServer,UDPServer,TCPServer.

python SocketServer documentation

I found in SocketServer.TCPServer Example

class MyTCPHandler(SocketServer.BaseRequestHandler):

        def handle(self):
            self.data = self.request.recv(1024).strip()

But self.request doesn't have recv method

If you have any solution to fix it or to change max limit of recv buffer.

It doesn't work, if the payload receive is larger than 8192(I suspected to be buffer limit).I will not be able to have the full payload in handle method. — user1778354, Aug 05 '13 at 22:15
It's possible that your `send` code is only sending the first 8K. Or that you've got an 8K network buffer somewhere along the way. But really, trying to send >8K messages over UDP is a bad idea. I'll write an answer to explain why, but the short version is: Unless you have a very good reason (and know what you're doing), don't try to get this to work; just use TCP. — abarnert, Aug 05 '13 at 22:29

score 4 · Accepted Answer · answered Aug 06 '13 at 20:54

Finally, I found it in baseserver python source code :

480 class UDPServer(TCPServer):
481 
482    """UDP server class."""
483
484    allow_reuse_address = False
485
486    socket_type = socket.SOCK_DGRAM
487
488    max_packet_size = 8192

My modified main (see server.max_packet_size )

if __name__ == '__main__':    
    server = SocketServer.UDPServer((HOST, LISTEN_PORT), MyUDPHandler)
    server.max_packet_size = 8192*2
    server.serve_forever()

score 0 · Answer 2 · answered Aug 05 '13 at 22:27

There are a number of factors involved here.

First, UDP's maximum payload size is 65535. However, the IPv4 header and UDP header count as part of that 65535, so practically speaking, the maximum user-payload size is 65504.

Second, you can't receive more than the smallest receive buffer at any level in the stack. I could explain how to get around this, but I won't, because…

Finally, any packet larger than a single MTU will get fragmented along the way. Unless the fragments just happen to arrive in exactly the same order they were sent, and without any sizable delay, they cannot be reassembled, so you will lose the entire packet. Even on a connection with only 1% Ethernet packet loss, you can expect more than a third of your large UDP packets to be lost. There are very few applications where this is acceptable.

So, what should you do? Well, there are three different answers:

Set up a dedicated network that can guarantee near-perfect reliability.
Write your own protocol to sit on top of UDP and break up large packets into MTU-sized packets.
Just use TCP instead of UDP.

There are some cases where option 2 makes sense—e.g., with many streaming A/V protocols, the audio frames, video iframes, and metadata are all under 1K, and can usually be dropped with no major consequence, so it's not worth the cost of using TCP just to deal with the keyframes, so instead they break the keyframes up into 1K chunks (e.g., rectangular sections of the window) or write explicit TCP-esque check-and-resend code just for keyframes, etc.

But usually, if you need to send large messages around, TCP is going to be a lot easier.

I know all that,but I fuzz proprietary protocol using up to 8 times classic MTU on UDP — user1778354, Aug 06 '13 at 21:00

python change max limit recv buffer in UDPServer

2 Answers2