1

My client-server application uses a basic implementation as provided by this MSDN article. However, the variation is that I am not using the <EOF> type of delimiter because the way my application works is this:

  1. Server waits for client
  2. Client connects
  3. Server waits for clients to request something
  4. Client eventually requests something (like "Please send CV of Hans Passant in PDF format?")
  5. Server replies appropriately
  6. Server waits for clients to request something (i.e. goes back to Step 3 again)

This cycle continues until the connection is closed.

In reference to this SO question, is it possible, via any of the .NET implementations of receiving data from a socket (Socket.Receive, TCP.GetStream().Read(...)...etc.) is it possible that there will be a time when a Read or Receive operation will return a false 0? (e.g. some data is read already, the next packet gets stucked in traffic temporarily and the Read/Receive operation does not know, so it just assumes there is nothing more to read and just returns zero).

My code below:

public void recieveClientData()
        {
            byte[] bs = new byte[1];// 'commStream' is NetworkStream of the socket    
            commStream.Read(bs, 0, 1);//blocks until something comes in 
            // Buffer for reading data 
            Byte[] bytes = new Byte[1024];
            using (MemoryStream m = new MemoryStream())
            {
                //scoop what we have yet
                m.Write(bs, 0, 1);

                //get the remaining
                int length;

                while ((length = commStream.Read(bytes, 0, bytes.Length)) != 0)
                {
                    m.Write(bytes, 0, length);
                }

                //At this point, 'm' is sent for asyncronously processing
                ...

                //wait for another data from this client
                recieveClientData();
            }  
        }

Now at the line while ((length = commStream.Read(bytes, 0, bytes.Length)) != 0), let us assume that the client is sending the server a PDF file of about 32Mb (so the server can save it for future retrieval), is it possible that when the line commStream.Read(bytes, 0, bytes.Length) is executing in a loop, a network lag is preventing that stream from having something to read, such that it returns 0 (meaning there is a network delay of some kind on a network packet but before that packet eventually makes it through and gets transported, commStream.Read has already read 0 and returned)

Note that I can't use delimiters here because transmitted data is not always string.

Community
  • 1
  • 1
Damilola Olowookere
  • 2,253
  • 2
  • 23
  • 33
  • Are you generally concerned or is there an actual problem at hand? – SimpleVar Dec 30 '14 at 16:21
  • Generally concerned. The software I am working on is to be deployed to users whose majority are impatient, inexperienced, ordinary users... so I want to be very safe :) – Damilola Olowookere Dec 30 '14 at 16:22
  • If the full message size is known in advance, you can start every message with a fixed-size Length (number of bytes) indicator, so you'll know what to expect. Client sends `32mb` + pdf file, and server reads `32mb` and keeps reading until received pdf file is 32mb. – SimpleVar Dec 30 '14 at 16:28
  • Read and Receive don't return false 0; 0 means "end of stream". It might *timeout*, but that is via an exception, not a return value. – Marc Gravell Dec 30 '14 at 16:35
  • How can I know the number of bytes that the 32Mb image file will fill in the byte array (and can I guarantee this on another system? i.e. Is it guaranteed that the 'fixed-size length (number of bytes) indicator' on my computer is the same on another?) – Damilola Olowookere Dec 30 '14 at 16:38
  • @MarcGravell I know. That was a mistake...but I hope you get the point I was trying to make with that. – Damilola Olowookere Dec 30 '14 at 16:40
  • 1
    A blocking read will return the requested number of bytes unless the connection is closed or some exception is thrown. A non-blocking read() may very likely return 0 if there is nothing in the queue. – DrKoch Dec 30 '14 at 16:41
  • @emmanuel The number of bytes it will fill is the number of bytes that it is... The fixed-size indicator is YOURS. You just agree with yourself to begin each message with 8bytes indicating the length of the whole message - aka how much to expect. – SimpleVar Dec 30 '14 at 16:42
  • 3
    @DrKoch This is wrong. _The Read operation reads as much data as is available, up to the number of bytes specified by the size parameter_. You can't expect `Read` to return the requested number of bytes. – ElderBug Dec 30 '14 at 17:17
  • RE: Knowing the # of bytes - You have to open the file with some kind of file stream to read and send it right? Use the [Length](http://msdn.microsoft.com/en-us/library/system.io.filestream.length%28v=vs.110%29.aspx) property to get the number of bytes you'll be sending. As others have stated, you can send this information (the number of bytes to expect) beforehand in a fixed number of bytes (Int64 is 8 bytes). Then you can simply increment a counter based on the number of bytes returned by `Read` and you'll know when you've received all the expected data for that file. – Idle_Mind Dec 30 '14 at 17:42
  • @DrKoch Does it mean it is possible that a thread can be blocked forever on `Socket.Receive()`? (take a scenario where there are 128 bytes data to read and I am reading in buffer size of 64 bytes, so first read scoops 64 bits, continues to next loop - since we typically do a `while(SocektObject.Receive(...) != 0 )` - and does a second read, now all 128 bytes read and goes for third loop...waiting indefinitely for something that will never come...)? – Damilola Olowookere Dec 30 '14 at 17:44
  • 1
    That's correct. If the connection isn't closed then it will just sit there at the Read() line waiting for something else to come in. It's up to **you** to decide that a complete "message" has been received (based on # bytes and/or type flags) and do something about it before going back to the blocking Read() call. – Idle_Mind Dec 30 '14 at 17:49
  • @ElderBug .. unless the requested number of bytes is 1, (sorry, pedantic troll mode today:). – Martin James Dec 30 '14 at 19:37

1 Answers1

3

TCP connections are stream oriented. There is no concept of message. Therefore is is very likely (at least on a bad network), that one of your Write() will be received with multiple Read(), or that multiple Write() will be received with a single Read(). You should never rely on Read() to break your stream in messages/requests.

The usual way is to know in advance what you will receive, or to send a size before actual data of unknown size. With TCP, data is guaranteed to be transmitted uncorrupted, so you should rely on this instead. Of course, a big problem can always occurs and reset the connection, but then you will have an exception, and handle it separately.

You should take a look at BinaryReader and BinaryWriter, these classes might make things simpler for you : a ReadString() will read a string complete, and a ReadBytes() will read exactly the number of bytes specified.

Edit: To clarify a bit, Socket.Receive() and NetworkStream.Read() with 64 as size parameter are not guaranteed to return with 64 bytes. They can return with 1, 10, 32, 63 or whatever (not 0, it would keep blocking, unless there is a timeout). BinaryReader.ReadBytes(64), on the other hand, is guaranteed to wait for 64 bytes. Connection problems are handled separately with exceptions.

ElderBug
  • 5,926
  • 16
  • 25
  • The data handled is not always string. So `ReadString` and all those variants can't help. Let's say I am sending a 32Mb image, How can I know the number of bytes that the 32Mb image file will fill in the byte array (and can I guarantee this on another system? i.e. Is it guaranteed that the 'fixed-size length (number of bytes) indicator' on my computer is the same on another?) – Damilola Olowookere Dec 30 '14 at 16:42
  • @emmanuel If your file is 2000 bytes long, it is 2000 bytes long, there is nothing more to add, and you use `ReadBytes(2000)`. If the size is not known beforehand, you can do ReadUInt64()` to read the size, and then `ReadBytes(size)`, or break it down in smaller chunk if it is really large. – ElderBug Dec 30 '14 at 16:48
  • I remember reading something about big-endian bla..bla... That is why I worry about: **...is it guaranteed that the 'fixed-size length (number of bytes) indicator' on my computer is the same on another?** – Damilola Olowookere Dec 30 '14 at 17:41
  • 1
    @emmanuel Since your file is just a bunch of bytes, it doesn't have endian, so you don't have to worry. For integers, like the file size, you can indeed send it as little-endian and read it as big-endian, and break everything. You can either handle the endian yourself or use `BinaryReader`/`BinaryWriter`, which handle it (and always use little-endian btw). – ElderBug Dec 30 '14 at 17:51
  • (for future individuals referencing this thread, please note that marking this entry as answer is not exhaustive. Do check comments made on the original question, especially those by Elderbug, Idle_Mind..etc.) – Damilola Olowookere Dec 30 '14 at 18:04
  • I've implemented my network com by storing the size of data in first 8 bytes followed by the data itself. Now at receiving end, I `Socket.Read()` into a `byte[] buffer` until at least the 8 bytes is read, then use `BitConverter.ToInt64` on first 8 bytes of `byte[] buffer` so that I can track when all data received. Now in this implementation of mine, I do not know where `BinaryReader/BinaryWriter` fits in so that I will not need to bother about bit order of my "prefix data size" across different computers. Please clarify. Thanks. – Damilola Olowookere Jul 03 '15 at 13:55
  • @emmanuel The idea is to use BinaryReader/Writer instead of direct socket Read/Write. You wrap the NetworkStream in a BinaryReader/Writer and use WriteInt64/ReadInt64. The advantage is that it will handle the reading/writing of exactly 8 bytes, and handle the endian. – ElderBug Jul 04 '15 at 08:27
  • Please any example code as to how to do that exactly? How to 'wrap' networkStream in BinaryReaser, since network sockets have r/w function calls with prototypes that do not accept binaryreader/writer. Thanks – Damilola Olowookere Jul 04 '15 at 17:27
  • can you attend to this please? – Damilola Olowookere Jul 05 '15 at 08:48
  • @emmanuel You just have to use `new BinaryReader(myNetworkStream)` – ElderBug Jul 05 '15 at 11:51
  • I changed `int lenBytesRcvdt = ssClientSocket.Client.Receive( tempDataBuffer, 1024, SocketFlags.None );` to `int lenBytesRcvdt = br.Read( tempDataBuffer, 0, 1024 );`. The br is defined as `BinaryReader br = new BinaryReader( ssClientSocket.GetStream() );`. Questions: `1. Is my implementation of BinaryReader correct?` and `2. Does the network stream get 'cleared' after every NtwrkSocketStream.Read(), such that for every round of Read() from the network socket stream, new data is always received starting at index 0 of the network stream and the stream pointer is also at 0?`. Thanks. – Damilola Olowookere Jul 12 '15 at 15:21
  • More specifically, is `int lenBytesRcvdt = br.Read( tempDataBuffer, 0, 1024 );` also going to block until something is received (like the normal direct Socket Read/Write). As regards writing to the stream, while there is `BinaryReader.ReadInt64;` there is no `BinaryWriter.WriteInt64` so I just used direct `BinaryWriter.Write` to write all the bytes needed to be sent. Is this right too? – Damilola Olowookere Jul 12 '15 at 15:37
  • @emmanuel This is not correct because `BinaryReader.Read()` doesn't is the same as `NetworkStream.Read()`. Use `BinaryReader.ReadBytes()` if you want to read a fixed amount of bytes. – ElderBug Jul 12 '15 at 16:26
  • @emmanuel And there is a `WriteInt64()`, but it's `Write(Int64)` (overloaded) – ElderBug Jul 12 '15 at 16:34
  • If I use `BinaryReader.ReadBytes()` to replace `NetworkStream.Read()`, how do I handle `block until all data from networkstream is received`? (Pls kindly take the pain to explain). For example, client may be sending `3453123 bytes`. So first `8 bytes` will store the value `3453123`; the next `4 bytes` stores an integer that correspond to a `com code` (e.g. `1`=`I am sending Alice's pdf`, `2`= `send me photo of ElderBug`, etc) and remaining `3453111` bytes will store the data if any (e.g. if `com code` is `1`, then Alice's pdf` is `3453111` bytes long [in some cases, this may be string]) – Damilola Olowookere Jul 12 '15 at 16:59
  • I mean given that `br.ReadBytes( tempDataBuffer, 0, 1024 )` is reading `1024` bytes at a time – Damilola Olowookere Jul 12 '15 at 17:01
  • @emmanuel If you want to read a file 3453111 bytes long, use `.ReadBytes(3453111)` (with the variable instead). To start really simple, you just need `size = br.ReadInt64(); code = br.ReadInt32(); fileData = br.ReadBytes(size);`. If you want to split the file because it might be really big, use sth like `.ReadBytes(1024*1024)` multiple times. – ElderBug Jul 12 '15 at 17:15
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/83075/discussion-between-emmanuel-and-elderbug). – Damilola Olowookere Jul 12 '15 at 17:22
  • Please follow up on my messages in chat. – Damilola Olowookere Jul 12 '15 at 17:47