inconsistent byte array read from socket

Question

I have two applications running on two different servers. One application connects to the other one by socket connection and through that socket it sends data. The receiving application reads that data and does something with it. The sending application uses the next piece of code to do it:

out.write(int2ByteArray(size));
out.flush();
Thread.sleep(5);
out.write(full_packet);
out.flush();

full_packet is a byte array with the dimension of size * 53 and int2ByteArray() method looks like this :

private static  byte[] int2ByteArray(int myInt){
    return ByteBuffer.allocate(4).order(ByteOrder.LITTLE_ENDIAN).putInt(myInt).array();
}

So basically, I send the size of my data packet before I send the actual data.

The receiving application reads the data using the next piece of code:

while (running){
    long startTime = System.currentTimeMillis();
    // read input data
    byte[] first_drop = new byte[4];
    in.read(first_drop);
    int size = byteArray2Int(first_drop);
    byte[] packet  = new byte [size * packet_size];
    Thread.sleep(5);
    in.read(packet);

    // do some thing with packet

    // go to sleep
    long endTime = System.currentTimeMillis();
    Thread.sleep(1000 - (endTime-startTime));
 }

packet_size equal 53. byteArray2Int() method looks like this:

private int byteArray2Int(byte[] bArr){
        return ByteBuffer.wrap(bArr).order(ByteOrder.LITTLE_ENDIAN).getInt();
    }

in is of type DataInputStream.

The problem starts now. When I run both applications on my laptop everything is fine and all data comes through the way it was sent. I run several threads of the sending application, sending several sizes of data streams. However, when run it on the servers, if the data is to big it does not come through as it should. If I am trying to send packets that are 5300 bytes long then it is fine. If I am sending packets that are 15900 bytes long and even bigger, at some point the reading application does not read the data packet properly and the next time it reads the 4 bytes of the size it receives some weird number and it jams the application. Same thing happens when I try to run several threads, each one send different data with a different size.

Can anyone point out the problem in the process? How can I fix it? Any suggestions? Any help would be appreciated.

Answer 1

InputStream doesn't always read all bytes arrived into your byte array. When the packet is small enough, the underlying InputStream implementation may return the whole packet, but, if it grows larger, as in your later experiments, it may not read them in a whole because its buffer can't hold that much of data, as mentioned by @ycesar. One way is to increase buffer size, but it is just hiding the problem. To really solve the problem, you need to check if the return value of InputStream.read() match the expected packet size.

There are convenience methods like IOUtils.read() in apache commons-io. You can make use of it or modify it for your own purpose as long as the license is followed. It looks like this:

public static final int EOF = -1;

public static int read(final InputStream input, final byte[] buffer, final int offset, final int length)
        throws IOException {
    if (length < 0) {
        throw new IllegalArgumentException("Length must not be negative: " + length);
    }
    int remaining = length;
    while (remaining > 0) {
        final int location = length - remaining;
        final int count = input.read(buffer, offset + location, remaining);
        if (EOF == count) { // EOF
            break;
        }
        remaining -= count;
    }
    return length - remaining;
}

Above code is copied from https://github.com/apache/commons-io/blob/2.5/src/main/java/org/apache/commons/io/IOUtils.java#L2901 licensed under Apache 2.0. No modifications.

Answer 2

You're ignoring the count returned by read(). The simplest way to accomplish this is wih DataInputStream.readFully().

Answer 3

I am not sure but maybe the default socket receive bufer is below expectations. Why don't you try with setReceiveBufferSize() function to increase this receive buffer size?

Hope it helps you.