Stack Overflow
Stack Overflow

Questions tagged [tpl-dataflow]

TPL Dataflow (TDF) is a .NET library for building concurrent applications. It promotes actor/agent-oriented designs through primitives for in-process message passing, dataflow, and pipelining. TDF builds upon the TPL (Task Parallel Library) in .NET 4 and integrates with async language support in C#, Visual Basic, and F#. TDF lacks join/merge by key (like SSIS) and time-based windowing (available in Rx).

Platform: .NET Framework 4.0 / .NET Core / .NET 6

Promotes: actor/agent programming model

Strengths: limiting parallelism (for example reading from disk bound vs. CPU compute bound tasks would be limited differently), data flow via message passing, integration with other .NET libraries like Rx and TPL.

Weaknesses: no data flow merge by key (like SSIS merge operation), and only the most trivial of time windowing (compare BroadcastBlock with Rx BufferWithTime() or CEP/StreamInsight windowing operators)

629 questions
0
votes
1 answer

Task.WhenAny acting like Task.WhenAll

I wrote a little program to test using BufferBlock (System.Threading.Tasks.Dataflow) to implement a dual-priority consumer-producer queue. The consumer should always use any items from the high-priority queue first. In this initial test, I have…
roki
  • 3
  • 1
0
votes
0 answers

TPL Dataflow - How to ensure one block of work is complete before moving onto next?

I'm still relatively new to TPL Dataflow, and not 100% sure if I am using it correctly or if I'm even suppose to use it. I'm trying to employ this library to help out with file-copying+file-upload. Basically the structure/process of handling files…
wpa
  • 220
  • 3
  • 11
0
votes
1 answer

Encapsulating with DataflowEx

I want to use DataflowEx to encapsulate this test. I am not sure that my design is correct as I have tried a lot to do it my self but I failed to make the Dataflow to complete as in my test and is not clear how I should post the server, IP…
Apostolis Bekiaris
  • 2,145
  • 2
  • 18
  • 21
0
votes
1 answer

Understanding TPL Dataflow, Blocks, and Continuation Tasks

I am working on a project where a need arose that is a perfect scenario for TPL Dataflow. Having relatively limited experience with it (and what I do have was done some time ago), I have been brushing up on it by reading Microsoft's documentation as…
JasCav
  • 34,458
  • 20
  • 113
  • 170
0
votes
0 answers

Dataflow (TPL): How randomly process consecutive data

Given This code represents some module/service The module receives a group of items each time and start process it The module should's wait for previous groups to complete to start processing the new one (only CPU limitations) The output of the…
Kamarey
  • 10,832
  • 7
  • 57
  • 70
0
votes
2 answers

Reduce CPU usage while processing large amount of data

I am writing a real time application which receives around 2000 messages per second which was pushed in a queue. I have written a background thread which process the messages in the queue. private void ProcessSocketMessage() { while…
Ramesh Durai
  • 2,666
  • 9
  • 32
  • 55
0
votes
0 answers

How TPL parallelism works?

I was experimenting with TPL blocks and came across a point where it puzzled me. TPL blocks shall run in parallel, depending on my settings (MaxDegreeOfParallelism). What I have done? I have created 5 blocks: Block 1 takes 3 seconds to…
Savaratkar
  • 1,974
  • 1
  • 24
  • 44
0
votes
0 answers

Parallelness within and across TPL Dataflow blocks?

I have some problems about the parallelness within and across TPL Dataflow blocks. Within block: I have a TransformBlock, with let's say MaxDOP of 4, performing things that I already know would benefit if it is as parallel as possible but the…
user1249190
  • 337
  • 3
  • 13
0
votes
1 answer

Better use of .net dataflow

I need help in better arranging the code in .NET TPL dataflow. here is the code var finalBlock = new ActionBlock(x => { List Resumes = new List(); using (var context = new…
user3522311
  • 21
  • 3
0
votes
0 answers

Task.WaitAll on a list of TransformBlock

I currently have a number of transform blocks running in a sequence a action task then does a wait all for these transform tasks come to completion however it looks very messy and as i add more transform blocks its going to get worse. Action…
Houlahan
  • 783
  • 3
  • 19
  • 46
0
votes
1 answer

Intel TBB computational graph: how to specify input queue capacity of the node

I'm looking for C++ analog of .NET TPL Dataflow library. In TPL Dataflow you can specify parallelism & blocks' capacity options. If the size of the block's input queue reaches it's capacity then the execution of the producer of the corresponding…
Dmitry Katkevich
  • 883
  • 7
  • 26
0
votes
1 answer

Continuous data stream using DataFlow and RX stops processing

I have setup a system to read messages from a queue indefinitely, then farm them off using Rx and TPL DataFlow. For some reason, after a few hundred messages the ActionBlock stops running hangs and I cannot figure out why. this.GetMessages()…
Chris
  • 26,744
  • 48
  • 193
  • 345
0
votes
1 answer

How to implement this specific Producer-Consumer pattern

I'm trying to write a windows service whose producers and consumers work like this: Producer: At scheduled times, get all unprocessed items (Processed = 0 on their row in the db) and add each one to the work queue that isn't already in the work…
user7127000
  • 3,143
  • 6
  • 24
  • 41
0
votes
2 answers

Delayed Post to DataFlow

When posting an item to a TPL DataFlow, is there any mechanism that can allow for a delayed post? public partial class BasicDataFlowService { private readonly ActionBlock workerBlock; public BasicDataFlowService() { …
Obsidian Phoenix
  • 4,083
  • 1
  • 22
  • 60
0
votes
1 answer

Retry tpl dataflow with/without Rx

I am using .net reactive extension with TPL dataflow. Here is my pipeline: I am getting data points as stream from some external source and then I am transforming data points using dataflow TransformBlocks. After this I am using Rx buffer to buffer…
Abhay
  • 928
  • 1
  • 10
  • 28
1 2 3
…
41 42