Socket connection between Java and Python - struct.unpack error

Question

I'm sending int by using DataOutputStream to Python via a socket but the data I receive at the Python part occasionally gets corrupted and I get the following error:

unpack requires a string argument of length 4

Not to mention that I run the Java part repeatedly. I have no idea why this happens. Please help.

Here is the Java part:

public static void main(String[] args) {

        Socket sock = null;
        DataOutputStream out = null;

        try {
            sock = new Socket("192.168.0.104", 1234);
            out = new DataOutputStream(sock.getOutputStream());

            out.writeInt(2);
            out.flush();
        } catch (IOException e) {
            e.printStackTrace();
        } finally {
            try {
                out.close();
                sock.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
}

And here is the Python part:

import socket
import struct

host = '192.168.0.104'
port = 1234
backlog = 5
size = 4
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((host, port))
s.listen(backlog)
while 1:
    client, address = s.accept()
    data = client.recv(size)
    unpacked = int(struct.unpack('>i', data)[0])
    if data:
        print unpacked
    client.close()

Here is the output:

2
2
2
Traceback (most recent call last):
  File "test_socket1.py", line 20, in <module>
    unpacked = int(struct.unpack('>i', data)[0])
struct.error: unpack requires a string argument of length 4

You're better off serializing data into formats suitable for network communication, like JSON/XML, while avoiding byte order, endianness, etc., otherwise, your approach will get messy! — raffian, Aug 21 '14 at 00:59
Can you add a try and except catch for that unpack line and when you do catch an exception, print the data out? That way you know what the data variable contains when your code throws the exception — ashwinjv, Aug 21 '14 at 00:59

score 2 · Accepted Answer · edited May 23 '17 at 12:29

From Python's documentation:

socket.recv(bufsize[, flags])

Receive data from the socket. The return value is a string representing the data received. The maximum amount of data to be received at once is specified by bufsize. See the Unix manual page recv(2) for the meaning of the optional argument flags; it defaults to zero.

The size you passed into recv is the maximum size. So at the point of reading, as long as there is data in the buffer, the function will return that number of bytes immediately. So you could be parsing a 3-byte string into an integer (which requires 4 bytes).

I suggest you check the length of bytes received. If less than 4, sleep a few milliseconds and try again. Finally, you concatenate all segments into one before parsing.

[EDIT] Found this post talking about something similar: Python socket recv from java client

Also, I need to correct what I said earlier. You don't need to sleep a few milliseconds (aka, explicitly calling time.sleep(a_few_milliseconds) , you can immediately call recv again, it will block until there is new data. If new data actually arrives in a few microseconds, you don't waste a few milliseconds in sleeping.

Socket connection between Java and Python - struct.unpack error

1 Answers1