0

I have a "single producer/single consumer" scenario implemented with new BlockingCollection of .NET 4.0.

The problem is that the producer thread awakes as soon as there's as a single space beocmes free in the collection. I want the producer to block until the consumer consumes at least half of the collection items. This is because producer speed is high and producing is expensive to the system.

How can I control the blocking condition for the producer?

Tim Lloyd
  • 37,954
  • 10
  • 100
  • 130
Xaqron
  • 29,931
  • 42
  • 140
  • 205

1 Answers1

3

An approach to consider is queuing fewer "bigger" items, rather than lots of "small" single items.

For example you could change the collection bound to 1 and change the item type to a list of items. In this way your producer could produce a list of 100 items and queue it, then the consumer would take this list and process it, leaving the producer to start on the next 100 items. The key here is that the producer will be optimized for producing larger batches of data in one batch before it is blocked waiting for the consumer to finish. There will be far less thrashing on the collection, but producing and consuming will still be overlapped.

When the consumer takes the list of 100 items, it is in effect taking half the total possible number of outstanding items i.e. 200 items. So conceptually this gives you your blocking condition.

Tim Lloyd
  • 37,954
  • 10
  • 100
  • 130
  • So why there's a `BoundedCapacity` ? They could hard-code it as 2 in the framework. The idea works but this is shifting the problem and a code like that is hard to maintain (Consider a change in list size) – Xaqron Jan 27 '11 at 08:50
  • @Xaqron Bounding is useful for overall throttling and also preventing fast production leading to memory issues - it's good for preventing runaway situations. I can't comment on how difficult it is to fit list semantics into your application, but my gut feeling that as an overall technique it is not too problematic. You just need to collect results before queuing, and enumerate the results when you dequeue. I'm guessing it's fine if producing and consuming is not always exactly balanced on half of the data, as it is not balanced at the moment. – Tim Lloyd Jan 27 '11 at 08:56