Stack Overflow
Stack Overflow

Questions tagged [tpl-dataflow]

TPL Dataflow (TDF) is a .NET library for building concurrent applications. It promotes actor/agent-oriented designs through primitives for in-process message passing, dataflow, and pipelining. TDF builds upon the TPL (Task Parallel Library) in .NET 4 and integrates with async language support in C#, Visual Basic, and F#. TDF lacks join/merge by key (like SSIS) and time-based windowing (available in Rx).

Platform: .NET Framework 4.0 / .NET Core / .NET 6

Promotes: actor/agent programming model

Strengths: limiting parallelism (for example reading from disk bound vs. CPU compute bound tasks would be limited differently), data flow via message passing, integration with other .NET libraries like Rx and TPL.

Weaknesses: no data flow merge by key (like SSIS merge operation), and only the most trivial of time windowing (compare BroadcastBlock with Rx BufferWithTime() or CEP/StreamInsight windowing operators)

629 questions
0
votes
2 answers

await SendAsync does not await on TPL Dataflow BatchBlock

The example program has the following BatchBlock: new BatchBlock(10, new GroupingDataflowBlockOptions { MaxNumberOfGroups = 2 });, to which there are 60 int data items being sent and on a separate task consumed. The issue is that the await…
dream3r
  • 45
  • 1
  • 6
0
votes
0 answers

Using SemaphoreSlim for tasks with return type

I am trying to create a producer/consumer TPL dataflow process. As part of it, I will be creating multiple producer tasks for different id range which will generate the data records for processing. So I am planning to throttle the number of threads…
askids
  • 1,406
  • 1
  • 15
  • 32
0
votes
0 answers

How does ordering and threading work in TPL DataFlow (Looking for some insight)

Intro While working on parallelizing an application for myself I noticed that when an item further in the past takes a long time eventually there will be only one running thread while the rest waits for this item to be completed so that the output…
Devedse
  • 1,801
  • 1
  • 19
  • 33
0
votes
1 answer

Reopen TPL Dataflow input after marking it complete

I'm trying to make a processing pipeline service that users can place an item into and wait for the results to finish being processed. My idea is to use DI to have it inject able. The problem I'm facing is that after processing the first set of…
Dylan Steele
  • 389
  • 3
  • 17
0
votes
2 answers

TPL DataFlow propagate completion only when all data has been processed

I have a producer sending data to a BufferBlock and when all data has been read from the source, it calls Complete(). The default behaviour is that when the completion is called, even if the buffer still has messages, it propagates the completion…
DWorkAccount
  • 46
  • 6
0
votes
1 answer

Is there a way to only process one task at a time per linked block in TPL Dataflow?

I have two classes, serviceclient, and a service. The serviceclient will generate messages that will be processed by the service. ServiceClient messages should be processed in a strictly FIFO order, with the next message only available when the…
cubesnyc
  • 1,385
  • 2
  • 15
  • 32
0
votes
1 answer

How to properly link several BufferBlocks together?

I would like several buffer blocks (producers) to dump into one bufferblock (consumer). I tried an extension of the code below, but the consumer is not being populated with any producer data. What am I doing wrong here? var bbA = new…
cubesnyc
  • 1,385
  • 2
  • 15
  • 32
0
votes
1 answer

Rx, TPL dataflow and windows service

Is it good idea to use Rx and dataflow with windows service working constantly? I just wondering to use rx interval to get data from db every 5 mintes and put it to blocking collection then use Tpl dataflow to process data. Is it correct way?…
dMilan
  • 257
  • 3
  • 13
0
votes
2 answers

Executing SQL select statements on a remote server using async/await + TPL dataflow: CPU or I/O bound?

My C# WinForms program runs on a Windows 7 client machine and submits SELECT queries to a SQL Server instance on a remote server (see DB code snippet below). I think I'm correct in categorizing the code that processes the query results (a series of…
VA systems engineer
  • 2,856
  • 2
  • 14
  • 38
0
votes
1 answer

C# DB query app causes the DB server to stop responding: What tools can I use on the DB server to identify and study the choke point?

I wrote a program modeled after the accepted answer for Throttling asynchronous tasks. See code below. My program is essentially an ad-hoc DB "text search" tool. It runs on a Win 7 client machine. The program composes a large collection of SELECT…
VA systems engineer
  • 2,856
  • 2
  • 14
  • 38
0
votes
1 answer

What is the exact difference between Task.ContinueWith and ActionBlock.LinkTo?

I am new to TPL Dataflow ActionBlock, TransformBlock etc. I used to practice Task.ContinueWith() to create a pipeline if needed. I recently started practicing about the TPL Dataflow and its blocks. But I am a bit confused about the exact difference…
SaddamBinSyed
  • 553
  • 3
  • 17
0
votes
0 answers

How to configure MaxDegreeOfParallelism BoundedCapacity MaxMessagesPerTask

I make a test with Walkthrough: Using BatchBlock and BatchedJoinBlock to Improve Efficiency, I got confused by MaxDegreeOfParallelism, BoundedCapacity, MaxMessagesPerTask, how to use them to improve the performance. I make a test with code below,…
Edward
  • 28,296
  • 11
  • 76
  • 121
0
votes
1 answer

Iterate IEnumerable selected from IGroupedObservable in RX

I have an IObservable sequence where T is a KeyValuePair which I group using GroupBy from System.Reactive.Linq. I would like to perform an aggregation operation on each of the IGroupedObservable>…
Jono
  • 1,964
  • 4
  • 18
  • 35
0
votes
1 answer

TPL Dataflow Complete Pipeline when condition matches

I thought it's very basic approach but I haven't found any example yet. I have one producer and one consumer and I want to finish the pipeline when at least x objects were processed. Additionally I need to know what objects have been…
BlackMatrix
  • 474
  • 5
  • 19
0
votes
1 answer

How to emit a cartesian product in TPL/Dataflow?

I am trying to implement the following behaviour: [TestMethod] public async Task ProducesCartesianProductOfInputs() { var block = new CartesianProductBlock(); var target = new BufferBlock>(); var left =…
Xavier Shay
  • 4,067
  • 1
  • 30
  • 54
1 2 3
…
41 42