Faster way to read from Socket with ByteBuffer?

Question

I have connected by TCP to a socket which is constantly sending a large amount of data, which I need to read in. What I have so far is a byte buffer that is reading byte by byte in a while loop. But the test case I am using right now is about 3 MB, which takes a while to read when reading in byte by byte.

Here is my code for this explanation:

ByteBuffer buff = ByteBuffer.allocate(3200000);
while(true)
{
     int b = in.read();
     if(b == -1 || buff.remaining() == 0)
     {
           break;
     }
     buff.put((byte)b);
}

I know that byte buffers are not thread safe and I'm not sure if this could be made faster by possibly reading in multiple bytes at a time and then storing it in the buffer? What would be a way for me to speed this process up?

Are you reading the same socket from multiple threads? Are you passing the filled buffer to another thread? Why do you mention thread safety at all? — erickson, Mar 26 '14 at 20:19
In any language (I know very little Java), if you only process read data 1 byte at a time, you're doing it wrong. Read in blocks, process the blocks 1 byte at a time. — San Jacinto, Mar 26 '14 at 20:24
@erickson I'm not using threads. I stated that because I have read elsewhere that ByteBuffers are not thread safe. But yes I am reading from a single socket. — G Boggs, Mar 26 '14 at 20:25
@SanJacinto That is what I was assuming as well haha. Do you know of some sort of standard of which to read in bytes. (By this I mean how many bytes should I read into one block at a time? Is there a standard amount?) — G Boggs, Mar 26 '14 at 20:27
It will largely depend on your application. In general, you read as much as your scenario will permit, and this differs from one scenario to another. — San Jacinto, Mar 26 '14 at 20:34
Why? Why can't you use either a SocketChannel or a byte[] array? — user207421, Mar 26 '14 at 21:25

score 1 · Accepted Answer · answered Mar 26 '14 at 20:28

1

Use a bulk read instead of a single byte read.

byte[] buf = new byte[3200000];
int pos = 0;
while (pos < buf.length) {
  int n = in.read(buf, pos, buf.length - pos);
  if (n < 0)
    break;
  pos += n;
}
ByteBuffer buff = ByteBuffer.wrap(buf, 0, pos);

Instead of getting an InputStream from the socket, and filling a byte array to be wrapped, you can get the SocketChannel and read() directly to the ByteBuffer.

answered Mar 26 '14 at 20:28

erickson

265,237
58
395
493

I believe this works, I'm running it now, but I'm not entirely sure how much faster it will be. But I have another question, is there a way to read from the socket without know the amount that will be sent? (In other words, an unknown buffer size) – G Boggs Mar 26 '14 at 20:38
@GBoggs Do you mean that you want to avoid having a hard-coded maximum, like 3200000 in this example? Instead, have memory allocated dynamically as needed to fit the data read from the socket? – erickson Mar 26 '14 at 20:43

user207421 · Answer 2 · 2014-03-26T22:54:41.947

There are several ways.

Use Channels.newChannel() to get a channel from the input stream and use ReadableByteChannel.read(buffer).
Get the byte[] array from the buffer with buffer.array() and read directly into that with in.read(array). Make sure the BB really does have an array of course. If it's a direct byte buffer it won't, but in that case you shouldn't be doing all this at all, you should be using a SocketChannel, otherwise there is zero benefit.
Read into your own largeish byte array and then use a bulk put into the ByteBuffer, taking care to use the length returned by the read() method.
Don't do it. Make up your mind as to whether you want InputStreams or ByteBuffers and don't mix your programming metaphors.

Faster way to read from Socket with ByteBuffer?

2 Answers2