0

Can TPL Dataflow be used for processing a TCP byte stream? I'm fairly new to this topic and am looking for the best way to process a TCP byte stream.

To be more specific: I'm using a protocol (HSMS) on top of TCP/IP which specifies messages with 4 length bytes and a message structure. I'm writing a library and have a consumer who receives the decoded messages. What I need to do in my library is:

  • read the bytes from the Socket and buffer them
  • as soon as I have the length bytes and the message itself I need to transform/decode it.
  • then I'll pass the message to the consumer of the library

Usually I have one producer and one consumer. And the IO operations are asynchronous.

I started doing it all manually but realized that it's quite complex and tedious, especially with thread safety. So I stumbled upon TPL dataflow and am curious if it's worth considering for this case. My main concern is that I have only one producer (Socket) and one consumer and wonder if it's overkill.

I also found System.IO.Pipelines but also there is not much information for my use case.

Josh
  • 287
  • 1
  • 8
  • Do you really get any improvements going parallel? I would just use Network Stream by itself. Link below I just used TCP and have a method ReadWrite to prevent reading and writing at same time. See : https://stackoverflow.com/questions/44471975/gps-socket-communication-concox – jdweng Jan 07 '20 at 15:23
  • You're not going to be able to have multiple threads simultaneously write to the same socket or you'll end up with overlapped messages unless you're absolutely certain that the messages will fit into a single buffer and the problem is that that can't be guaranteed. I can't rate Pipelines with Tasks enough - let me know if you want an example.w – Rowan Smith Jan 08 '20 at 10:13

1 Answers1

-1

I understand it's an old post.

Yes TPL dataflow is well suitable for this task, but for such a simple processing scenario, you might use system.io.pipeline within RX. It's quite direct once you made the effort to dig into io.pipeline and RX.

Then, Dataflow will come after that, connected to your IObservable<T>, as streaming pipeline you can use to elaborate complex data processing.

Regards

Guillaume Pelletier
  • 121
  • 4
  • Compared to TPL Dataflow, RX has a steeper learning curve. So for someone having no experience with neither, telling them to learn both doesn't make much sense. – Theodor Zoulias Aug 04 '20 at 15:41
  • LOL, As I see, still a lot of trolling here... Downvoting for a personal point of view doesn't make much sense, – Guillaume Pelletier Aug 04 '20 at 17:48
  • My vote represents my personal point of view. Which is that your answer points the OP to the wrong direction. They are already concerned about adding complexity to their app by using TPL Dataflow, and your suggestion essentially is: *add TPL Dataflow and RX and Pipelines and whatnot*. – Theodor Zoulias Aug 04 '20 at 18:02