How to consume an array of BlockingCollection, one element at a time, in C#?

Question

I've seen lots of examples of how to consume a BlockingCollection<T> in the producer-consumer scenario, even how to consume one element at a time here. I'm quite new to parallel programming though, so I'm facing the following problem:

The problem, indeed, is how to write the method ConsumerProducerExample.ConsumerMethod in the example below, such that it consumes the first 2 double of the BlockingCollection<Double> for every element in the array, then proceeds to consume the next 2 double for every element in the array and so on.

I have written the method in the example below, but it is not the way I want it to work. It is what I know how to do based in the exampled I linked above though. It is exactly the opposite way: as it is here, it will consume every double before jumping to the next BlockingCollection<Double> of the array. And, again, I want it to consume only two double, then jump to the next BlockingCollection<Double>, consume two double, etc. After completing the array loop, it shall proceed to consume the next two double for each BlockingCollection<Double> element again, and so on.

public class ConsumerProducerExample
{
    public static BlockingCollection<Double>[] data;
    public static Int32 nEl = 100;
    public static Int32 tTotal = 1000;
    public static Int32 peakCounter = 0;

    public static void InitData()
    {
        data = new BlockingCollection<Double>[nEl];
        for (int i = 0; i < nEl; i++) data[i] = new BlockingCollection<Double>();
    }

    public static void ProducerMethod()
    {
        Int32 t = 0, j;
        while (t < ConsumerProducerExample.tTotal)
        {
            j = 0;
            while (j < ConsumerProducerExample.nEl)
            {
                data[j].Add(Math.Sin((Double)t / 10.0D));
                j++;
            }
            t++;
        }
        j = 0;
        while (j < ConsumerProducerExample.nEl)
        {
            data[j].CompleteAdding();
            j++;
        }
    }

    public static void ConsumerMethod()
    {
        // THE PROBLEM IS WITH THIS METHOD

        Double x1, x2;
        Int32 j = 0;
        while (j < Program.nEl)
        {
            while (!data[j].IsCompleted)
            {
                try
                {
                    x1 = data[j].Take();
                    x2 = data[j].Take();
                }
                catch (InvalidOperationException)
                {
                    break;
                }

                if (x1 * x2 < 0.0)
                {
                    Program.peakCounter++;
                }
            }

            j++;
        }
    }
}

They should be used like this:

ConsumerProducerExample.InitData();
Task task1 = Task.Factory.StartNew(ConsumerProducerExample.ProducerMethod);
Task task2 = Task.Factory.StartNew(ConsumerProducerExample.ConsumerMethod);

Any suggestions?

In short, this is a try to count peaks in the solutions of nEl Differential Equations concurrently (the solutions are represented by sin(x) in this example).

BTW, why are you using `while` loops to iterate a range of indexes? `for` loops are better for that. And in some cases, you're iterating a collection, so `foreach` would be even better. — svick, Oct 21 '12 at 10:39
Well, when handling random access collections (like arrays), I prefer using `while` loops. I did some basic tests with `for`, `foreach` and `while` (the exactly same code inside of the loops `for` and `while` -- like accessing an element and doing some basic logical operation with it), and the `while` performed faster... — Girardi, Oct 21 '12 at 18:40
I quite doubt that, `for` loops should be faster than `while` loops, if anything. In any case, it's most likely premature optimization unless you measured that it makes a difference in this specific case. — svick, Oct 22 '12 at 18:18

svick · Answer 1 · 2012-10-22T18:16:46.187

2

Basically, you need to get rid of your inner loop and add another outer loop, that iterates over all of the collections, until they're complete. Something like:

while (!data.All(d => d.IsCompleted))
{
    foreach (var d in data)
    {
        // only takes elements from the collection if it has more than 2 elements
        if (d.Count >= 2)
        {
            double x1, x2;
            if (d.TryTake(out x1) && d.TryTake(out x2))
            {
                if (x1 * x2 < 0.0)
                    peakCounter++;
            }
            else
                throw new InvalidOperationException();
        }
    }
}

edited Oct 22 '12 at 18:16

answered Oct 21 '12 at 10:47

svick

236,525
50
385
514

May I edit you answer to make it fit exactly into the problem? The appropriete method is `data.All(d => d.IsCompleted)`, so it waits for all the `BlockingCollection` to finish. Thank you! – Girardi Oct 21 '12 at 21:34
@Girardi Done, with some modifications. – svick Oct 22 '12 at 18:17

How to consume an array of BlockingCollection, one element at a time, in C#?

1 Answers1