0

I am trying to set up a concurrent queue that will enqueue data objects coming in from one thread while another thread dequeues the data objects and processes them. I have used a BlockingCollection<T> and used the GetConsumingEnumerable() method to create a solution that works pretty well in simple usage. My problem lies in the facts that:

  1. the data is coming in quickly, data items being enqueued approximately every 50ms
  2. processing each item will likely take significantly longer than 50ms
  3. I must maintain the order of the data items while processing as some of the data items represent events that must be fired in the proper order.

On my development machine, which is a pretty powerful setup, it seems the cutoff is about 60ms of processing time for getting things to work right. Beyond that, I have problems either with having the queue grow continuously (not dequeuing fast enough) or having the data items processed in the wrong order depending on how I set up the processing with regard to whether/how much/where I parallelize. Does anyone have any tips/tricks/solutions or can point me to such that will help me here?

Edit: As pointed out below, my issue is most likely not with the queuing structure itself so much as it is with trying to dequeue and process the items faster. Are there trick/tips/etc. for portioning out the processing work so that I can keep dequeuing quickly while still maintaining the order of the incoming data items.

Edit (again): Thanks for all your replies! It's obvious I need to put some more work into this. This is all great input, though and I think it will help point me in the right direction! I will reply again either with a solution that I came up with or a more detailed question and code sample! Thanks again.

Update: In the end, we went with a BlockingCollection backed by a ConcurrentQueue. The queue worked perfectly for what we wanted. In the end, as many mentioned, the key was making the processing side as fast and efficient as possible. There is really no way around that. We used parallelization where we found it helped (in some cases it actually hurt performance), cached data in certain areas, and tried to avoid locking scenarios. We did manage to get something working that performs well enough that the processing side can keep up with the data updates. Thanks again to everyone who kicked in a response!

Dennis
  • 215
  • 1
  • 3
  • 14
  • Is the data coming in in order or do you have to sort it? – Doug Dawson Apr 13 '15 at 14:36
  • 2
    Have you looked into third party projects like RabbitMQ? These have often been optimized and refined over time to make them extremely fast at throughput. – druidicwyrm Apr 13 '15 at 14:41
  • The data is coming in order. No need to sort. I haven't looked into 3rd party products as we have a lot of restrictions in place as far as approvals for 3rd party stuff. I will take a look, but my first preference is for code solutions. Thanks! – Dennis Apr 13 '15 at 14:47
  • Well, I'm actually not necessarily looking for another queue. You're right - it is more of a processing issue. I guess my question would be better asked from that perspective - are there some tricks I can use on the processing side to portion/parallelize the work to be able to dequeue faster but still maintain the order of the data items. – Dennis Apr 13 '15 at 15:11
  • Obviously, if data comes in faster, than it becomes processed, you'll get growing queue unless you'll make data processing in parallel. – Dennis Apr 13 '15 at 15:12
  • Sounds like you want to put processed events to some sort of ordered queue to preserve original order irrespective in what order parallel processing of incoming events completed... Otherwise if you must process in serial manner there is no way to keep up with faster incoming rate. – Alexei Levenkov Apr 13 '15 at 15:12
  • _"I must maintain the order of the data items while processing as some of the data items represent events that must be fired in the proper order"_ -- given that your cost of processing exceeds the per-unit cost you can accept, the requirement to process the items serially seems impossible. If you cannot introduce concurrency _somewhere_ (e.g. maybe not _all_ processing units have to be serialized?), you can't fix this. – Peter Duniho Apr 13 '15 at 15:14
  • "Powerful setup" probably means multi core. Simply working on the items sequentially will probably occupy only one core with whatever needs to be done. Perhaps you can parallelize the computing but preserve the order of the results (printing, messaging, whatever needs to be externally visible after processing) through some clever mechanism, e.g. by having a second queue for results. – Peter - Reinstate Monica Apr 13 '15 at 15:15
  • 2
    _"Are there trick/tips/etc. for portioning out the processing work"_ -- there's no "trick". You just need a design that allows enough concurrency for the consuming to keep up with the producing. Again, if you can limit your serialization to just those units of processing where it's required, and there is a significant number where it's not, that might address the problem. But without [a good, _minimal_, _complete_ code example](http://stackoverflow.com/help/mcve) that clearly illustrates the scenario, no specific answer can be provided. – Peter Duniho Apr 13 '15 at 15:17

3 Answers3

2

If you are using TPL on .NET 4.0, you can investigate the TPL Dataflow library simple usage, as this library (it's not a third party, it's a library from Microsoft being distributed via NuGet) provide the logic which saves the order of data being processed in your system.

As I understand, you got some data which will come in order, which you have to mantain after some work at each of data item. You can use for this TransformBlock class or BufferBlock linked with ActionBlock: simply put the data on it's input, set up the action you need to be run on each item, and link this block with classes you need (you even can make it IObservable to create a responding UI.

As I said, TPL Dataflow blocks are incapsulating FIFO queue logic, and they are saving the order for results on their action. And the code you can write with them is multithreading-oriented (see more about maximum degree of parallelizm in TPL Dataflow).

VMAtm
  • 27,943
  • 17
  • 79
  • 125
  • From the linked MSDN article - _"The TPL Dataflow Library (System.Threading.Tasks.Dataflow namespace) is not distributed with the .NET Framework 4.5."_ It is a Microsoft product however. – Gusdor Apr 13 '15 at 15:23
  • Thanks! I am using .NET 4.0 but it looks like the TPL Data Flow is targeted at .NET 4.5? – Dennis Apr 13 '15 at 15:30
  • You can get this library via NuGet, as like many more Microsoft Products. – VMAtm Apr 13 '15 at 15:30
  • @Dennis https://www.nuget.org/packages/Microsoft.Tpl.Dataflow/4.0.0 - you can try out this version, but major version is oriented for the `4.5` – VMAtm Apr 13 '15 at 15:32
0

I think that you are okay with the blocking queue. I enqueue thousands of messages per second in a BlockingCollection and the overhead is very small.I think you should do the following:

  • Add a synchronized sequence number when enqueuing the messages
  • Use multiple consumers to try to overload the queue

In general focus on the processing time. The default collection type for BlockingCollection is ConcurrentQueue, so the default is that the it is a FIFO (First in, first out) queue, so something else seems to be wrong.

weismat
  • 7,195
  • 3
  • 43
  • 58
0

some of the data items represent events that must be fired in the proper order.

Then you may differentiate dependent items and process them in order while processing other items in parallel. Maybe you can build 2 separate queues, one for items to be processed in order, dequeued an processed with a single thread and another dequeued by multiple threads.

We need to know more about input and expected processing.

Guillaume
  • 12,824
  • 3
  • 40
  • 48