Unwanted nul characters preceding bytes from TCP socket read using DataInputStream

Question

I'm writing an Android app that involves connecting to a TCP server (that I also wrote) and sending/receiving text from it. Right now I have a bug in my final read (client-side).

When I use the debugger in Eclipse, it shows that I am receiving all the bytes that are sent, but for certain pieces of text, if I'm expecting n bytes, I'll get the first n - k, some m NUL bytes, and then the final k - m meaningful bytes. If I'm interpreting the problem correctly, Java is seeing the huge amount of 0s and deciding that there's nothing useful to be read after (the debugger shows the byte array and the string that it translates to, but then throws it away if I try to inspect it further).

How can I ignore the massive influx of NUL and just read the important stuff?

// Find out how many bytes we're expecting back
int count = dis.readInt(); // dis is a DataInputStream
dos.writeInt(count); // dos is a DataOutputStream

// Read that many bytes
byte[] received = new byte[count];
int bytesReceived = 0;
int bytesThisTime = 0;
while (-1 < bytesReceived && bytesReceived < count) {

    bytesThisTime = dis.read(received, 0, count);
    if (bytesThisTime <= 0) break;

    bytesReceived += bytesThisTime;
    String bytesToString = new String(received, 0, bytesThisTime, "UTF-8");
    sb_in.append(bytesToString);
    received = new byte[count];

}
in = sb_in.toString();

Here is the server code that is doing the writing:

            // Convert the xml into a byte array according to UTF-8 encoding
            // We want to know how many bytes we're writing to the client
            byte[] xmlBytes = xml.getBytes("UTF-8");
            int length = xmlBytes.length;

            // Tell the client how many bytes we're going to send
            // The client will respond by sending that same number back
            dos.writeInt(length);
            if (dis.readInt() == length) {
              dos.write(xmlBytes, 0, length); // All systems go - write the XML
            }

            // We're done here
            server.close();

To aid in debugging, you might either add logging to the receiver so you know precisely what chunks it got each time through the loop and what it did with them and/or use packet monitoring software (tcpdump/windump) to log the actual packets sent between the pieces. Manual testing of either end against an open source program called netcat can also be very helpful. — Chris Stratton, Dec 01 '12 at 19:51

Tomasz Nurkiewicz · Answer 1 · 2012-12-01T19:11:34.433

0

Replace:

String bytesToString = new String(received, "UTF-8");

with:

String bytesToString = new String(received, 0, bytesThisTime, "UTF-8");

Basically dis.read(received, 0, count) can return any number of bytes between 0 and count. bytesThisTime tells you how many bytes were read this time. But later you use whole array rather than only the part that was actually read.

BTW consider using InputStreamReader which will decode string on the fly for you (but count will have different semantincs then). Moreover read carefully through IOUtils API.

edited Dec 01 '12 at 19:11

answered Dec 01 '12 at 19:05

Tomasz Nurkiewicz

334,321
69
703
674

I'm still getting the problem. I'm reading the number of bytes the server says I should be getting, but a lot of them are 0, and I can't seem to figure out why. I'm adding the server code above. – bibismcbryde Dec 01 '12 at 19:26
On a second look, it seems that the problem is that java is accounting for this huge amount of NUL bytes as part of the expected number of bytes, so it's actually reading the right amount but the majority of them shouldn't be there. I'm not sure what could be causing that though. – bibismcbryde Dec 01 '12 at 19:36

score 0 · Answer 2 · answered Dec 02 '12 at 00:38

Java is seeing the huge amount of 0s and deciding that there's nothing useful to be read after

No. Java doesn't look at the data at all, let alone make semantic decisions like that.

How can I ignore the massive influx of NUL and just read the important stuff?

There is no 'massive influx of NUL' to ignore. Java doesn't do that, TCP doesn't do that, nothing does that.

There are just programming errors in your own code.

I could detail those endlessly, but in essence you should be using DataInoutStream.readFully() instead of trying to replicate it with your own bug-ridden version.

Unwanted nul characters preceding bytes from TCP socket read using DataInputStream

2 Answers2