Tuning buffer length for reading small data from NetworkStream

Question

How to fine tune the bufferSize while reading small data from the TcpClient/NetworkStrem? If the bufferSize is big like 1024, 4096 the Read/BeginRead blocks. If I set the bufferSize to 16, 32 it works without blocking.

Does setting the bufferSize to 1 guarantee there won't be any blocks? Will the performance impact be very bad?

It looks like setting the ReadTimeout to values like 1000, 2000 has no effect over blocking. Is there any other way to make the blocking be short? (NoDelay = true doesn't work)

public static IObservable<byte[]> AsyncReadChunk(this Stream stream, int bufferSize)
{
    var buffer = new byte[bufferSize];

    return Observable.FromAsyncPattern<byte[], int, int, int>(stream.BeginRead, stream.EndRead)(buffer, 0, bufferSize)
        .Select(cbRead =>
                    {
                        var dataChunk = new byte[cbRead];

                        Buffer.BlockCopy(buffer, 0, dataChunk, 0, cbRead);

                        return dataChunk;
                    });
}

public static IObservable<byte[]> AsyncRead(this NetworkStream stream, int bufferSize)
{
    return Observable.Defer(() => stream.DataAvailable ? AsyncReadChunk(stream, bufferSize) : Observable.Return(new byte[0]))
        .Repeat()
        .TakeWhile((dataChunk, index) => dataChunk.Length > 0);
}

How do you read your data? The docs state, that NetworkStream.Read will not block but return the value 0 when no data is available. — Jan, Nov 25 '11 at 14:10
I read the data with the code above. I loop and read the data in blocks of bufferSize while the NetworkStream.DataAvailable is true. — Samet S., Nov 25 '11 at 14:13
Can you show the complete code involved in reading from the socket? — Jan, Nov 25 '11 at 14:28
@Jan : Really? I would have thought (like all other types of stream) than NetworkStream.Read would return zero to indicate the end of the stream, and block (or not invoke EndRead in the async case) until data was available. I've got a **lot** of code relying on this behaviour. Can you share your source? — spender, Nov 25 '11 at 14:46
@Spender: Yes, i don't believe it too. Here is the link: http://msdn.microsoft.com/en-us/library/system.net.sockets.networkstream.read.aspx. See the section remarks. — Jan, Nov 25 '11 at 14:58
@Spender, the wording seems to be ambiguous i think. The docs of Stream.Read, the docs of NetworkStream.Read .NET 1.1 (http://msdn.microsoft.com/en-us/library/system.net.sockets.networkstream.read(v=VS.71).aspx) and yours and mine experience tells that it will block when no data is available and the connection is still established. — Jan, Nov 25 '11 at 20:09

Jan · Answer 1 · 2011-11-26T07:35:29.710

It looks like setting the ReadTimeout to values like 1000, 2000 has no effect over blocking.

Msdn says, that ReadTimeout has no effect when using the async BeginRead method.

Does setting the bufferSize to 1 guarantee there won't be any blocks?

No, of cause not. When not a single byte is sent over the connection, the Read call will block regardless of the buffersize.

Will the performance impact be very bad?

I guess you are talking about the 1 byte buffer here. It depends on the amount and frequency of data you receive and what code you execute on EndRead. When processing a stream with a high bandwidth, the impact can be big. You have to try and watch your cpu while receiving data.

I'm not really sure what you want to achieve respectively what your concern about blocking is.

When you start a Receive (or Networkstream.Read) with a buffer of say 1024 bytes, and you receive 10 Bytes on the socket, the Read call will return that 10 Bytes after a short delay but will not block until the whole buffer is filled.

Is there any other way to make the blocking be short?

What do you mean with short. As i say, even with a big buffer, the Read won't block endlessly when receiving small amounts of data.

(NoDelay = true doesn't work)

That a completely different story but is might be interesting to set it to true on your senders side (if you are in control of that, too).

When set to false (the default), it will combine small datachunks being sent into bigger ones to reduce the overhead of one tcp packet (40 Bytes header).

EDIT

I mean NetworkStream.BeginRead to return immediately if there is no data.

What about using stream.DataAvailable? It should return false when there is no data.

In addition, when using the async pattern, isn't it expected behaviour that the calls will block until there is something to do? Otherwise you will get active polling in a busy loop.

When the buffer is big it sometimes waits 60 seconds to return (when it is not fully filled or filled to some amount)

Hm, can't believe that. What kind of data are you sending over the channel? 1 byte every minute? 1000 bytes per second?

>>I'm not really sure what you want to achieve respectively what your concern about blocking is.
I mean NetworkStream.BeginRead to return immediately if there is no data. When the buffer is big it sometimes waits 60 seconds to return (when it is not fully filled or filled to some amount). — Samet S., Nov 26 '11 at 05:22

Tuning buffer length for reading small data from NetworkStream

1 Answers1