3

I have a strange question that involves NetworkStreams in C# .Net Core 2.2.101

My setup is as follows:

  • I have a list of meters
  • Each meter has a list of registers (a register saves a value, for example: the voltage or current)
  • Meters are connected to a GSM modem via RS485 (irrelevant for the question)
  • I send commands to the modem to read specific registers for specific meters

For each register of each meter, I send a command using stream.Write(bytesToSend, 0, bytesToSend.Length); to ask the meter to send me the data that is saved in a specific register in the meter. Directly after sending, I read the response using stream.Read(buffer, 0, buffer.Length);. I also set a read timeout of 5 seconds, which will block and wait for 5 seconds before moving on to the next register if no response has been received before the timeout.

The problem:

What is happening is that when I ask the meter for data, sometimes it will take too long and the timeout will be reached, after which I will move on to ask the next register for data, but then sometimes the first register will reply with data after I have moved on to the next register (meaning that the NetworkStream now has the data from the previous register). Since I have already moved on in my for-loop, my program thinks that the data I am reading from the stream is for the current register, when in fact it is for the previous register from the previous iteration. This messes up the data that goes into the database, because I am saving the wrong value for the wrong register.

My question is: Is there a clever way to ignore any incoming data from a previous iteration in my for-loop? Unfortunately there is no information in the data received that could be used to identify for which register the data is for.

Here is a snippet of what my write and read requests look like:

stream.ReadTimeout = 5000;
stream.WriteTimeout = 2000;

foreach (Meter meter in metersToRead)
{
  foreach (Register register in meter.Registers)
  {
    // Write the request to the meter
    stream.Write(bytesToSend, 0, bytesToSend.Length);

    // Read response from meter
    requestedReadingDataCount = stream.Read(buffer, 0, buffer.Length);

    // Extract the value from the response buffer and save the value to the database
    ...
  }
}

I want to try and clone the stream and use the cloned stream for communication regarding the current register iteration, so that when a response comes in after I have closed the cloned stream and moved on to the next register, the response will fail since the stream has been closed. However, I am not sure if you can clone a C# NetworkStream? Anybody know?

My last resort will be to make a call to the database after I read the data for each register, to check if the data I received is reasonable for that register, but I fear that this might slow the program down with all the database calls and I would have to build up some rules that will determine whether a value is reasonable for the current register.

Any ideas would be greatly appreciated. If you have any questions, please let me know and I will try my best to explain it further.

Edit

Here is an updated code snippet, as well as an image that will better explain the issue that I am having.

private async Task ReadMeterRegisters(List<MeterWithRegisters> metersWithRegisters, NetworkStream stream)
{
    stream.ReadTimeout = 5000; /* Read timeout set to 5 seconds */
    stream.WriteTimeout = 2000; /* Write timeout set to 2 seconds */

    foreach (Meter meter in metersToRead)
    {
          foreach (Register register in meter.Registers)
          {
                // Instantiate a new buffer to hold the response
                byte[] readingResponseDataBuffer = new byte[32];

                // Variable to hold number of bytes received
                int numBytesReceived = 0;

                try
                {
                    // Write the request to the meter
                    stream.Write(bytesToSend, 0, bytesToSend.Length);

                    // Read response from meter
                    numBytesReceived = stream.Read(buffer, 0, buffer.Length);
                }
                catch (IOException) /* catch read/write timeouts */
                {
                    // No response from meter, move on to next register of current meter
                    continue;
                }

                // Extract the value from the response buffer and save the value to the database
                ...
          }
    }
}

Description of code snippet

  • You need to explain your scenario better, and provide a good [mcve] that reproduces the problem (a challenge when dealing with networking code, especially code that normally interacts with some custom environment not available to the general public, but just as essential, if not more so, when asking a question here as for any other debugging question). It seems odd to me that you are using the same stream for each device you are querying. Do you in fact use the same underlying socket as well? If so, what mechanism does your protocol have for distinguishing devices (meter) when reading/writing? – Peter Duniho Mar 12 '20 at 17:13
  • Thanks will keep that in mind. I use the modbus protocol, so each response from a meter will contain the meter ID of the meter that is responding. So for every modem that connects, I open a new socket to read its connected meters. – Morné Lombard Mar 12 '20 at 17:20
  • _"each response from a meter will contain the meter ID of the meter that is responding"_ -- without a decent code example, I can't post an answer. However, given what you say, IMHO you should create a new communications layer in your code that separates traffic to and from each meter, based on its ID, into separate streams. Basically: having been presented with a difficult problem to solve, turn it into a simpler problem with a known solution. – Peter Duniho Mar 12 '20 at 17:36
  • Related to all that: your code currently appears to synchronously query each meter in succession. Is that _really_ necessary? Does the modbus protocol (which I don't know much about) require that you restrict communications to a single device on the bus at a time? Because if not, I wouldn't bother with the sequential querying, but instead send out all of the queries all at once, and then organize them as responses are returned according to the meter ID contained within the response. ... – Peter Duniho Mar 12 '20 at 17:40
  • ... Implicit in that suggestion is the assumption that the protocol affords a way to detect message boundaries, something the code you posted appears to not concern itself with, even though it almost certainly should. – Peter Duniho Mar 12 '20 at 17:40
  • Yes I have to do it synchronously for each register, because the response I get does not contain the register number that the data in the response is for. I wish it did, because then I would not have this issue. [Here is more info on how a modbus packet will look](http://www.simplymodbus.ca/FC03.htm) – Morné Lombard Mar 12 '20 at 18:03
  • Under what circumstances is the response delayed? Is there ever a situation where a delayed response would be sent in a different order than the requests were sent? Is there ever a situation where a request will not yield a response at all, but a subsequent request would? I ask these questions, because so far your description seems to suggest the protocol uses TCP, but behaves in a very un-TCP-like way. ... – Peter Duniho Mar 12 '20 at 19:04
  • ... When dealing with TCP, a timeout or other failure generally means that you have to abandon the connection altogether. Conversely, if you don't abandon the connection, the presumption is that if you wait long enough, you'll eventually get all of the data you expected, in exactly the order you expected. – Peter Duniho Mar 12 '20 at 19:04
  • It would be good if you could be a lot more detailed about the exact configuration of your whole system. Exactly what do the slave devices (meters) look like, how are they communicating, do you have any intermediate master/server device, etc.? Looking at the web site you referenced, it states that for modbus over TCP, the client includes a 16-bit "transaction identifier" in a request, which is echoed as part of the response. I would think you should be using that to keep requests and responses organized. – Peter Duniho Mar 12 '20 at 19:13

1 Answers1

1

It sounds like the issue here is that in a timeout scenario, the read operation is still completing at some point in the future but writing to the old buffer. If that is the case, perhaps the simplest option is to not reuse the read buffer in the event of a timeout (meaning: assign a new byte[] to buffer), and consider that network-stream burned (since you now can't know what the internal state is).

An alternative approach would be to not read until you know there is data; you can't do that from NetworkStream, but on Socket you can check .Available to see whether there is data to be read; that way, you won't be performing ambiguous reads. You can also perform a zero-length read on a socket (at least on most OS-es); if you pass a zero-length buffer, it will block until either the timeout or until data becomes available, but without consuming any data (the idea being that you follow a zero-length read with a non-zero-length read if you find that data has become available).

In the more general case: you might find you get better throughput here if you switch to asynchronous IO rather than synchronous IO; you could even use the array-pool for the buffers. For dealing with large volumes of connections, async IO is almost always the way to go.

Marc Gravell
  • 1,026,079
  • 266
  • 2,566
  • 2,900
  • _"on Socket you can check .Available to see whether there is data to be read; that way, you won't be performing ambiguous read"_ -- IMHO you should elaborate on this, because I don't understand how that would address the problem. The OP is intentionally abandoning the read after a timeout period. If data does not become available during that period, but is then returned by the earlier-commanded sensor ("meter") after they move on to the next, they wind up with the same problem they had in the first place. ... – Peter Duniho Mar 12 '20 at 17:10
  • ... More generally, I have never seen a scenario where checking the `Available` property on a socket made things better. – Peter Duniho Mar 12 '20 at 17:10
  • @PeterDuniho when combined with zero-length-reads, it absolutely can be used effectively; you are essentially doing buffer-free async waits, then using `Available` to validate the state; but yes, there are lots of ways it can be used very incorrectly – Marc Gravell Mar 12 '20 at 20:26
  • Maybe you've run into a situation where a _"buffer-free async wait"_ is helpful. I haven't. In my experience, the best approach to network I/O is to use IOCP. In .NET, that's any of the async methods. With sockets, using the methods ending with `...Async` affords an improvement due to less GC overhead. But, waiting on a socket without a buffer ready to accept data just means that by the time your code gets around to trying to read from the socket, the network layer might have discarded the data it buffered earlier, forcing extra overhead onto the network itself. ... – Peter Duniho Mar 12 '20 at 21:25
  • ... In any case, I don't see how any of that helps in the situation above. It's TCP, so ultimately there's nothing the receiving end can do to change the order or values of bytes which will be received. Changing _when_ the data is read doesn't help that situation at all. – Peter Duniho Mar 12 '20 at 21:25
  • @Peter again, it is contextual; I have a socket server that usually has about a half million open connections over a tiny number of nodes (you're probably using it right now); you *really really* don't want to reserve buffers in managed code for all those potential async reads until you need to (because there is data to be read) – Marc Gravell Mar 12 '20 at 21:51
  • That's fine. But I still don't see how that in any way applies the question here. – Peter Duniho Mar 13 '20 at 02:02
  • 1
    @PeterDuniho the question is *specifically* about reading a non-trivial number of sockets and avoiding problems where delayed (timed-out) reads end up overwriting a buffer that is being shared between multiple reads; what we're talking about here is ways of side-stepping the issue by not putting yourself in a position where a failed read (with access to a buffer to corrupt) is possible; how is that *not* relevant? – Marc Gravell Mar 13 '20 at 07:59
  • _"how is that not relevant?"_ -- I didn't see anything in the question that suggests that the number of sockets is at issue. There appears to be a single connection, with some kind of timeout implemented, and a failure to handle that correctly. That said, if your suggestion helps, that's great. I'm just saying that, as someone who is at least reasonably familiar with socket I/O, I'm not comprehending how the suggestion I asked about would help, and so your post bears _at least_ some further explanation. If you don't want to improve the post, that's your prerogative...I'm just trying to help. – Peter Duniho Mar 13 '20 at 15:57