How to correctly use BlockingCollection.GetConsumingEnumerable?

Question

I'm trying to implement a producer/consumer pattern using BlockingCollection<T> so I've written up a simple console application to test it.

public class Program
{
    public static void Main(string[] args)
    {
        var workQueue = new WorkQueue();
        workQueue.StartProducingItems();
        workQueue.StartProcessingItems();

        while (true)
        {

        }
    }
}

public class WorkQueue
{
    private BlockingCollection<int> _queue;
    private static Random _random = new Random();

    public WorkQueue()
    {
        _queue = new BlockingCollection<int>();

        // Prefill some items.
        for (int i = 0; i < 100; i++)
        {
            //_queue.Add(_random.Next());
        }
    }

    public void StartProducingItems()
    {
        Task.Run(() =>
        {
            _queue.Add(_random.Next()); // Should be adding items to the queue constantly, but instead adds one and then nothing else.
        });
    }

    public void StartProcessingItems()
    {
        Task.Run(() =>
        {
            foreach (var item in _queue.GetConsumingEnumerable())
            {
                Console.WriteLine("Worker 1: " + item);
            }
        });

        Task.Run(() =>
        {
            foreach (var item in _queue.GetConsumingEnumerable())
            {
                Console.WriteLine("Worker 2: " + item);
            }
        });
    }
}

However there are 3 problems with my design:

• I don't know the correct way of blocking/waiting in my Main method. Doing a simple empty while loop seems terribly inefficient and CPU usage wasting simply for the sake of making sure the application doesn't end.

• There's also another problem with my design, in this simple application I have a producer that produces items indefinitely, and should never stop. In a real world setup, I'd want it to end eventually (e.g. ran out of files to process). In that case, how should I wait for it to finish in the Main method? Make StartProducingItems async and then await it?

• Either the GetConsumingEnumerable or Add is not working as I expected. The producer should constantly adding items, but it adds one item and then never adds anymore. This one item is then processed by one of the consumers. Both consumers then block waiting for items to be added, but none are. I know of the Take method, but again spinning on Take in a while loop seems pretty wasteful and inefficient. There is a CompleteAdding method but that then does not allow anything else to ever be added and throws an exception if you try, so that is not suitable.

I know for sure that both consumers are in fact blocking and waiting for new items, as I can switch between threads during debugging:

EDIT:

I've made the changes suggested in one of the comments, but the Task.WhenAll still returns right away.

public Task StartProcessingItems()
{
    var consumers = new List<Task>();

    for (int i = 0; i < 2; i++)
    {
        consumers.Add(Task.Run(() =>
        {
            foreach (var item in _queue.GetConsumingEnumerable())
            {
                Console.WriteLine($"Worker {i}: " + item);
            }
        }));
    }

    return Task.WhenAll(consumers.ToList());
}

Answer 1

GetConsumingEnumerable() is blocking. If you want to add to the queue constantly, you should put the call to _queue.Add in a loop:

public void StartProducingItems()
{
    Task.Run(() =>
    {
        while (true)
            _queue.Add(_random.Next());
    });
}

Regarding the Main() method you could call the Console.ReadLine() method to prevent the main thread from finishing before you have pressed a key:

public static void Main(string[] args)
{
    var workQueue = new WorkQueue();
    workQueue.StartProducingItems();
    workQueue.StartProcessingItems();

    Console.WriteLine("Press a key to terminate the application...");
    Console.ReadLine();
}