Stack Overflow
Stack Overflow

Questions tagged [tpl-dataflow]

TPL Dataflow (TDF) is a .NET library for building concurrent applications. It promotes actor/agent-oriented designs through primitives for in-process message passing, dataflow, and pipelining. TDF builds upon the TPL (Task Parallel Library) in .NET 4 and integrates with async language support in C#, Visual Basic, and F#. TDF lacks join/merge by key (like SSIS) and time-based windowing (available in Rx).

Platform: .NET Framework 4.0 / .NET Core / .NET 6

Promotes: actor/agent programming model

Strengths: limiting parallelism (for example reading from disk bound vs. CPU compute bound tasks would be limited differently), data flow via message passing, integration with other .NET libraries like Rx and TPL.

Weaknesses: no data flow merge by key (like SSIS merge operation), and only the most trivial of time windowing (compare BroadcastBlock with Rx BufferWithTime() or CEP/StreamInsight windowing operators)

629 questions
0
votes
1 answer

C# TPL: Invoke method on outer scoped instance

So my title was fairly obscure, here is what I'm worried about. Can I invoke a method on an instance of a class that is declared outside of the block without suffering pitfalls i.e Are there concurrency issues for code as structured…
Black Dynamite
  • 4,067
  • 5
  • 40
  • 75
0
votes
1 answer

Process all file change events within specified time frame with TPL Dataflow

I am monitoring multiple log files across multiple directories. I need to trigger an SSIS package when a file has fired an onchange event. Easy enough, but the complication is I don't want to trigger the SSIS package every time there is a change on…
JAGuiD
  • 13
  • 1
0
votes
0 answers

async Throttling using AsyncCollection or BufferBlock from the TPL in .net

I would like to read a stream and buffer it's output so that a consumer could read it before the producer has finish full reading of the stream. example, read from an Http stream and forward to another ... This using the .net4.5 TPL library. I…
Cedric Dumont
  • 1,009
  • 17
  • 38
0
votes
0 answers

TPL DataFlow - Completion only being called on first run

I am trying to create my first TPL DataFlow service in a Windows Form application using .Net 4.5. In overview, the application loads and parses some data from a .csv text file and, after some processing/interrogation generates a SQL database record…
Neilski
  • 4,385
  • 5
  • 41
  • 74
0
votes
1 answer

Cancellation not returning past whenall using httpclient

Here is an updated version of the code posted under httpclient.GetStringAsync blocking The question is when cancel is done, though the tasks are cancelled, I am expecting Await Task.WhenAll(tasks) to return and print whats in finally, but its not. I…
Kallol
  • 264
  • 1
  • 12
0
votes
3 answers

Access three lists with three threads, print the items in order

Here is the question: Say there are 3 lists l1, l2 & l3 of same length. Three threads accessing three lists. Say T1 -> l1, T2 ->l2 & T3 ->l3. It should print in the order say first element of 1st then first element of 2nd list and then first…
Hui Zhao
  • 655
  • 1
  • 10
  • 20
0
votes
0 answers

Correct approach and scenario for TPL-dataflow?

I have a scenario that I've tried to solve with TPL. The result is decent but I'm wondering if my design is flawed and if there is room for any improvement or radical changes. Scenario: An user can "subscribe" to X number of items and can set a…
rosaflodhest
  • 1
  • 1
0
votes
2 answers

Concurrently running incorrectly, provide more channels than expected

I have a application, which to make concurrently task running. Here we set MaxDegreeOfParallelism=4, which means at any time at most 4 tasks running concurrently. In this case, I only have 4 channels available. Otherwise an exception Could not get…
user1108948
0
votes
1 answer

PC Queue using TPL?

Original How would I create a Producer-Consumer Queue in C# leveraging the TPL and/or Concurrent Collections? I'm using .NET 4.5+. Here's my first attempt: public class SampleFileProcessor { private readonly BlockingCollection _queue = …
Lee Grissom
  • 9,705
  • 6
  • 37
  • 47
0
votes
1 answer

TPL dataflow bufferblock message send on a timer

I want to build on this question. So far I have got a way to process jobs in parallel. I'm running this in a console app. I get say 50 jobs from db, process them using TPL DataFlow and so far so good. But I realized that if there is a job that takes…
Alex J
  • 1,547
  • 2
  • 26
  • 41
0
votes
1 answer

Use events to forward exceptions for producer/consumer using Dataflow

I’m trying to implement a producer/consumer queue using Dataflow for HTTP requests towards a web service. I found an excellent post from Stephen Cleary, which is covering exactly this scenario. However, in contrast to Stephen’s post, I cannot mark…
rene
  • 1,618
  • 21
  • 26
0
votes
2 answers

Should items flowing in a TPL Dataflow network be DTO or POCO?

(Now that I have your attention with acronyms ...) Maybe a better way to ask the question is: When should you use DTOs and when POCOs in a TPL Dataflow network? (Because the better choice may depend on circumstances). I've done it both ways and I'm…
davidbak
  • 5,775
  • 3
  • 34
  • 50
0
votes
1 answer

Architecture suggestions for serial comms based instrument control application

I have just started a project to control an industrial instrument that communicates via RS-232 serial port. Baud rate is limited to 9600...fairly slow. This instrument can operate in two modes - polled, where the PC sends a request, and the…
Tom Bushell
  • 5,865
  • 4
  • 45
  • 60
0
votes
1 answer

How to do async 'paged' processing of EF select result

I'm writing something that loads records from SQL server onto an azure queue. The thing is, the number of items in the select result might be very large, so I would like to start the queuing stuff while data is still being retrieved. I'm trying to…
Jochen van Wylick
  • 5,303
  • 4
  • 42
  • 64
0
votes
1 answer

MaxDegreeOfParallelism blocks the Main thread?

I have the next code, and works fine: private void BtBasicIntroClick(object sender, EventArgs e) { var stopwatch = new Stopwatch(); stopwatch.Reset(); stopwatch.Start(); var executionDataflowBlockOptions = new…
user1229323
1 2 3
…
41 42