Socket reading using BufferedInputStream

Question

I'm using Java's BufferedInputStream class to read bytes sent to a socket. The data to the socket is HTTP form so generally is a header with defined content-length, then some content.

The problem I'm having is that sometimes BufferedInputStream.read() will not read the full amount of data sent to it. It returns the number of bytes read but this is much less than has been sent. I have verified the bytes sent with Wireshark and can confirm the full message is being transmitted.)

Sample code below:

BufferedInputStream inFromClient = new BufferedInputStream(socket.getInputStream());
int contentLength = getContentLengthFromHeader();    
byte[] b = new byte[contentLength];
int bytesRead = inFromClient.read(b, 0, contentLength);

Once read() is finished sometimes bytesRead is equal to contentLength but on other occasions read() does not seem to read as far as the end of the content. Does anyone have any ideas on what is happening? Is Java buffering output? Are there better ways of reading from sockets?

Answer 1

You're assuming that read() fills the buffer. Check the Javadoc. It transfers at least one byte, that's all it says.

You don't need both a large buffer and a BufferedInputStream. Change the latter to DataInputStream.readFully().

Answer 2

This is normal behavior for the read() method: you need to keep reading in a loop until read returns -1. (see http://docs.oracle.com/javase/7/docs/api/java/io/BufferedInputStream.html#read(byte[],%20int,%20int))

In general, it happens because the read method is trying to return all the data it can to you before blocking, not all the data you will ever get.

There are a couple of utility methods I frequently use for this sort of thing: (snipped out of context - note that I am not the author of the channelCopy method, but the source is attributed)

   /**
    * Efficiently copy from an InputStream to an OutputStream; uses channels and 
    * direct buffering for a faster copy than oldCopy. 
    * @param in - non-null readable inputstream
    * @param out - non-null writeable outputstream
    * @throws IOException if unable to read or write for some reason.
    */
   public static void streamCopy(InputStream in, OutputStream out) throws IOException {
      assert (in != null);
      assert (out != null);
      ReadableByteChannel inChannel = Channels.newChannel(in);
      WritableByteChannel outChannel = Channels.newChannel(out);
      channelCopy(inChannel, outChannel);
   }

   /**
    * Read the *BINARY* data from an InputStream into an array of bytes. Don't 
    * use this for text.
    * @param is - non-null InputStream
    * @return a byte array with the all the bytes provided by the InputStream 
    * until it reaches EOF.
    * @throws IOException 
    */
   public static byte[] getBytes(InputStream is) throws IOException{
      ByteArrayOutputStream os = new ByteArrayOutputStream();
      streamCopy(is, os);
      return os.toByteArray();
   }


   /**
    * A fast method to copy bytes from one channel to another; uses direct 16k 
    * buffers to minimize copies and OS overhead.
    * @author http://thomaswabner.wordpress.com/2007/10/09/fast-stream-copy-using-javanio-channels/
    * @param src - a non-null readable bytechannel to read the data from
    * @param dest - a non-null writeable byte channel to write the data to
    */   
   public static void channelCopy(final ReadableByteChannel src, final WritableByteChannel dest) throws IOException {
      assert (src != null);
      assert (dest != null);
      final ByteBuffer buffer = ByteBuffer.allocateDirect(16 * 1024);
      while (src.read(buffer) != -1) {
         // prepare the buffer to be drained
         buffer.flip();
         // write to the channel, may block
         dest.write(buffer);
         // If partial transfer, shift remainder down
         // If buffer is empty, same as doing clear()
         buffer.compact();
      }

      // EOF will leave buffer in fill state
      buffer.flip();

      // make sure the buffer is fully drained.
      while (buffer.hasRemaining()) {
         dest.write(buffer);
      }
  }