EndReceive never returns a 0, how do you know when packets are finished in tcp/ip?

Question

This is similar to my last question. I'm making a simple tcp/ip chat program and I'm having a little difficulty with the EndReceive Callback function. I pasted in Microsoft's implementation (see below) and what I've noticed is that if the message has been read in it's entirety it won't run the callback function until the next message is sent i.e. EndReceive will never return a 0. For example if I send a text message of 5 characters and the buffer size is 20, 'read' will be 5, I'll process the string and call BeginReceive, however BeginReceive doesn't run until the server sends another message. The else section is never reached because s.EndReceive never returns a 0. How would I know when data has been received completely?

public static void Read_Callback(IAsyncResult ar){
 StateObject so = (StateObject) ar.AsyncState;
 Socket s = so.workSocket;

 int read = s.EndReceive(ar);

 if (read > 0) {
            so.sb.Append(Encoding.ASCII.GetString(so.buffer, 0, read));
            s.BeginReceive(so.buffer, 0, StateObject.BUFFER_SIZE, 0, 
                                  new AsyncCallback(Async_Send_Receive.Read_Callback), so);
 }
 else{
      if (so.sb.Length > 1) {
           //All of the data has been read, so displays it to the console
           string strContent;
           strContent = so.sb.ToString();
           Console.WriteLine(String.Format("Read {0} byte from socket" + 
                           "data = {1} ", strContent.Length, strContent));
      }
      s.Close();
 }
}

Answer 1

EndReceive may well get a 0 if the stream is closed and all data has been consumed. However, for most other purposes there is no logical break in a stream - and even NetworkStream.DataAvailable only tells you what is available at the client - not what was sent by the server. Hence: if you are sending multiple messages / etc on the same stream it is usually necessary to either prefix each block with the number of bytes expected, or to use some delimiter sequence that would not normally be expected in the middle of content. The former is often easier if the content could be arbitrary binary content (i.e. it is tricky to predict a sequence that isn't expected), but the latter allows streaming without knowing the lengths ahead of time (useful if you are computing the content on-the-fly, as otherwise you may need to buffer the content first to know the length (or do a dry run discarding the data).

Also note that Read is not guaranteed to return all the available data - it would be entirely legitimate to send "HELLO", and have Read return just 1 (the H). Hence you should expect to have to combine multiple Read calls into a single message (again, using the length-prefix or the expected delimiter sequence).

In this scenario, I'd be tempted to send HELLO\0 (where \0 is a zero-byte). Then I would consume via Read until either it returns <=0, or a \0 is found. In the \0 case, anything buffered before the terminator represents a single message; anything after the delimiter should be stored and processed as part of the next message-or-messages. As a test case, imagine you get Read return 13, with sequence a\0b\0cde\0f\0\0\0g, which which should "a", "b", "cde", "f", "", "", "g" (noting that the g should not be processed - it might be part of a longer sequences that isn't yet received - buffer until you get a \0).