The are at least two issues you need to decide on:
- How to exchange messages?
- In what format?
Regarding 1. TCP sockets are the lowest level and you would need to deal with low level things like recognizing messages boundaries. Also, TCP connection gives you reliable delivery but only as long as the connection is not reset (due to for example a temporary network failure). If you want your application to gracefully recover when a TCP connection resets, you need to implement some form of messages acknowledgements to keep track what needs to be resend over the new connection. OMQ gives you higher level of abstraction than plain TCP connection. You don't need to deal with a stream of bytes but with whole messages. It still does not give you reliable delivery, messages can get lost, but it gives several communication patterns that can be used to ensure reliable delivery. 0MQ is also highly performant, IMO it is a good choice.
Regarding 2, if interoperability with other languages is not needed, Pickle is a very convenient and Pythonic choice. If interoperability is needed, you can consider JSON, or, if performance is an issue, binary format, such as Google protocol buffers. This last choice would require the most work (you'll need to define messages formats in .idl files) this would definitely not feel Pythonic.
Take a look how exchange of messages (any serializable Python object) over a plain socket can look like:
def send(sockfd, message):
string_message = cPickle.dumps(message)
write_int(sockfd, len(string_message))
write(sockfd, string_message)
def write_int(sockfd, integer):
integer_buf = struct.pack('>i', integer)
write(sockfd, integer_buf)
def write(sockfd, data):
data_len = len(data)
offset = 0
while offset != data_len:
offset += sockfd.send(data[offset:])
Not bad, but as you can see having to deal with serialization of a message length is quite low level.
And to receive such message:
def receive(self):
message_size = read_int(self.sockfd)
if message_size == None:
return None
data = read(self.sockfd, message_size)
if data == None:
return None
message = cPickle.loads(data)
return message
def read_int(sockfd):
int_size = struct.calcsize('>i')
intbuf = read(sockfd, int_size)
if intbuf == None:
return None
return struct.unpack('>i', intbuf)[0]
def read(sockfd, size):
data = ""
while len(data) != size:
newdata = sockfd.recv(size - len(data))
if len(newdata) == 0:
return None
data = data + newdata
return data
But this does not gracefully deal with errors (no attempt to determine which messages were delivered successfully).
