Stack Overflow
Stack Overflow

Questions tagged [tpl-dataflow]

TPL Dataflow (TDF) is a .NET library for building concurrent applications. It promotes actor/agent-oriented designs through primitives for in-process message passing, dataflow, and pipelining. TDF builds upon the TPL (Task Parallel Library) in .NET 4 and integrates with async language support in C#, Visual Basic, and F#. TDF lacks join/merge by key (like SSIS) and time-based windowing (available in Rx).

Platform: .NET Framework 4.0 / .NET Core / .NET 6

Promotes: actor/agent programming model

Strengths: limiting parallelism (for example reading from disk bound vs. CPU compute bound tasks would be limited differently), data flow via message passing, integration with other .NET libraries like Rx and TPL.

Weaknesses: no data flow merge by key (like SSIS merge operation), and only the most trivial of time windowing (compare BroadcastBlock with Rx BufferWithTime() or CEP/StreamInsight windowing operators)

629 questions
0
votes
0 answers

TPL Data flow giving .Net referenced assemblies in output

I am using Visual studio 2013. I added Microsoft.TPL.Dataflow 4.5.24 from Nuget Package Manager solution for project in VS. I also having Setup for this project. When i rebuild the Setup Project All the .net referenced libraries(System.IO,…
user2478122
  • 55
  • 1
  • 7
0
votes
1 answer

Wrapping my head around TPL DataFlow

I have a windows service (using Topshelf) which hosts a WCF service (created using SimpleInjectorServiceHost) and also spins up threads (using Task.Run). The idea is to: WCF service receives commands post those commands to BufferBlock. The…
user2321864
  • 2,207
  • 5
  • 25
  • 35
0
votes
1 answer

Parallel pipeline with ordered inputs and outputs

I have a pretty simple task to accomplish in C#. The user gives me a stream representing a large binary file (tens of GB in size). The file consists of lots and lots of distinct blocks. I need to read each block; perform some CPU-intensive analysis…
Bugmaster
  • 1,048
  • 9
  • 18
0
votes
0 answers

TPL Dataflow: push elements bunch from within block

What is the best way to accomplish the following in TPL dataflow: I have a TransformManyBlock that for each received element may produce a very long sequence of output elements It would be nice to have an ability to push some elements from the…
Pavel Baravik
  • 173
  • 1
  • 9
0
votes
2 answers

How to stop processing pipeline on faulty block?

How can I stop processing of DataFlow blocks if one of the blocks made decision that an error occurred, preventing next blocks to run. I thought a block can throw an exception, but not sure what is the proper way to stop further processing…
Pablo
  • 28,133
  • 34
  • 125
  • 215
0
votes
1 answer

TPL Dataflow with Railway Programming

I'm attempting to implement a moderately complex TPL Dataflow pipeline with Railway Programming, where messages can get sent down the "happy path" if they succeed in the current block, or go to the failure block if they don't succeed. The problem…
BPiep
  • 9
  • 3
0
votes
0 answers

Using Rx C# create parallel filtering over large datasets

I am writing a search mechanism using Rx in C#. I have a dataset of 50k records which needs to run for search of keyword when user starts typing. I have created below code for that but I believe there are lots of issue related to concurrency and…
Balraj Singh
  • 3,381
  • 6
  • 47
  • 82
0
votes
0 answers

Propagate data from many UDP sockets to TPL Dataflow using async/await

I have to listen to several udp sockets and receive multicast datagrams. After receiving from some socket, data has to be sent to the TransformBlock. I don't want to copy received data for each received packet, so data buffer is sent to the…
shda
  • 729
  • 7
  • 19
0
votes
1 answer

Search implementation using Rx.Net

I am creating a search page implementation using Rx in C#. I have created a generic search method to search a keyword and add list of result on UI. Below is the code: Generic Search Method: public static IObservable…
Balraj Singh
  • 3,381
  • 6
  • 47
  • 82
0
votes
0 answers

Database async query and processing

Let me rephrase. I have a method that generates strings(paths) after given a start string(path) IF those paths are for a directory I want to enqueue that in the input of the method. After processing the path synchronously, I want to get the Data and…
Git
  • 214
  • 5
  • 16
0
votes
1 answer

Task.WhenAll doesn't resume

Below code is simplified version what i am trying to do. The code supposed to reach Console.ReadKey() line very quickly but it never happens. Task.WhenAll never resume next line. What is wrong on the code ? class Program { static void…
Freshblood
  • 6,285
  • 10
  • 59
  • 96
0
votes
1 answer

Application Dispatcher doesn't update and doesn't give errors

I have an application and i need to download a lot of files. Actually to download i use this code: System.Net.ServicePointManager.DefaultConnectionLimit = Int32.Parse(Properties.Settings.Default["download_threads"].ToString()); var block = new…
giogiowefj
  • 103
  • 7
0
votes
2 answers

Implementation issue with TransformBlock using TPL Dataflow

I am learning TPL dataflow. I tried creating a sample where I am posting some values from different Tasks and expecting the result back in the same Task to process it further. But the result is coming wrong. Following is my code. Let me know what…
Balraj Singh
  • 3,381
  • 6
  • 47
  • 82
0
votes
0 answers

TPL Dataflow Producer/Consumer with BroadcastBlock - tracking execution

Here's my scenario: multiple producers, single consumer. The consumer works synchronously, and should only process the last input (even abort processing of what is currently being processed and start with new input if any). So I've connected a…
user3566056
  • 224
  • 1
  • 12
0
votes
1 answer

TPL Dataflow Batchblock check for elements in Input buffer

My DataFlow pipeline starts with a BatchBlock. I would like to trigger the BatchBlock with the help of the TriggerBatch() method. My batch sizes can be variable, so when creating the Batchblock I have placed a rather high BatchSize which I don't…
Ricky
  • 81
  • 11
1 2 3
…
41 42