Socket Programming in Python

Question

I am working on a proof of concept using python that emulates a server/client communication using sockets to send UDP packets. I can easily do a simple client to server and back to client comms, but I am trying to introduce a "middle-man" into that communication. Conceptually the problem can be descirbed as, if "Joe" is the main client, he will send a message to "Steve" who is the middle man who will do something with that message before sending it to "Carol" who acts as the server that will process the new message and send a response back to the middle-man, "Steve". Eventually the middle-man will then send that message on elsewhere, but at the moment I am not worrying about that.

My current code looks like:

"Joe" (original client) looks like

# create dgram udp socket
try:
    s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
except socket.error:
    print ('Failed to create socket')
    sys.exit()

host = 'localhost'
port = 8888

print("start comms")
while 1:

    arr = ['Dog', 'cat', 'treE', 'Paul']
    num = random.randrange(0,4)
    #Send the string
    s.sendto(arr[num].encode(), (host, port))

"Steve" (middle man) looks like

host = ''
hostRT = 'localhost'
portVM = 8888
portRT = 8752

# socket to receive from "Joe"
s1 = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s1.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s1.bind((host, portVM))

# socket to send to "Carol"
s2 = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)   

print("start comms")

while 1:

    # receive from "Joe"
    data = s1.recvfrom(1024)
    num = data[0].decode()
    addrVM = data[1]

    # print data from Joe
    print(num)
    # add some excitement to Joe's message
    num += '!!!'

    # show received message address + port number
    print ("message[" + addrVM[0] + ":" + str(addrVM[1]) + ']')

    # Send to "Carol"
    s2.sendto(num.encode(), (hostRT, portRT))

    # receive from "Carol"
    d = s2.recvfrom(1024)
    reply = d[0].decode()
    addrRT = d[1]

    # show received message address + port number
    print ("message[" + addrRT[0] + ":" + str(addrRT[1]) + ']')

    # show Carol's response
    print ('Server reply : ' + reply)     

s1.close()
s2.close()

"Carol" (server) looks like

host = ''
port = 8752

s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
print ("socket created")
s.bind((host, port))

print ("Socket bind complete")

while 1:
    d = s.recvfrom(1024)
    data = d[0].decode()
    addr = d[1]

    print(data)

    reply = "Upper case client data = " + data.upper()

    print(reply)

    s.sendto(reply.encode(), addr)

    print ("message[" + addr[0] + ":" + str(addr[1]) + '] - ' + data.strip())

s.close()

Currently I can receive a message from Joe but then it hangs on the sending to the server Carol. I'm completely new to socket programming so any help would be greatly appreciated.

Edit for clarification

Using Python 3.4

Joe is sending packets non stop as to emulate the real life application that this proof of concept is for. Joe will be sending packets at a rate of roughly 1 packet / 4ms, but I am only concerned with the most recent packet. However, since the average turn around time for the round trip from Steve to Carol is around 10ms, I had originally thought to cache Joe's most recent packet in a local memory location and overwrite that location until Steve is ready to send a packet to Carol once she has responded with the last packet. However, for this simple proof of concept I haven't tried to implement that. Any suggestions on that would also be helpful.

Answer 1

There are multiple faults that contribute to the overall failure, some of which are not apparent (i.e. it sort of works until it crashes down somewhere else).

First of all, at the moment sends packets as fast as he cans. That alone can lead to significant packet loss everywhere else (that might be a good thing, since you now have to make sure your code survives packet loss). Unless you truly want to stress the network, something like time.sleep(0.1) would be appropriate in the send loop.

More importantly, steve's socket setup is all messed up. He needs two sockets at the most, not three. The way it is currently set up, carol answers steve to the IP address and port she got the packet from (which is quite sensible), but steve reads on a distinct socket that never gets data sent to. To make matters worse, the port steve's s3 listens on is actually the same one that carol uses! Since you are not using multicast. You can simply remove every reference to s3 from the code and use s2 exclusively.

Another problem is that you don't deal with packet loss. For example, if a packet gets lost between steve and carol, the code

# Send to "Carol"
s2.sendto(num.encode(), (hostRT, portRT))

# receive from "Carol"
d = s2.recvfrom(1024)  # s3 in the original

will hang forever, since Carol does not send any new packets after the one that got lost. As mentioned before, packet loss is way more likely since joe is blasting out packets as fast as he can.

To detect packet loss, there are a few options:

• Use multiple threads or processes for sending and receinv. This is going to make your program way more complex.
• Switch to asynchronous / non-blocking IO, in Python with the high-level asyncore or the more low-level select.
• Set and correctly handle socket timeouts. This is probably the easiest option for now, but is quite limited.
• Switch to TCP if you actually need reliable communication.

---

Apart from the aforementioned network problems, there are also some potential problems or inaccuracies:

• If you are using Python 2.x, the behavior of decode and encode on strings depends on the system configuration. Like many other potential problems, this has been fixed in Python 3.x by mandating UTF-8 (in 2.x you have to explicitly request that). In your case, that's fine as long as you only send ASCII characters.
• while(1) : looks really strange in Python - why the whitespace after the argument, and why parentheses . Why not while 1: or while True:?

• You can use tuple unpacking to great effect. Instead of

  data = s1.recvfrom(1024) num = data[0].decode() addrVM = data[1]

how about:

data, addrVM = s1.recvfrom(1024)
num = data.decode('utf-8')