Streaming Data BlockingCollection

Question

On page 88 of Stephen Toub's book

http://www.microsoft.com/download/en/details.aspx?id=19222

There is the code

private BlockingCollection<T> _streamingData = new BlockingCollection<T>();
// Parallel.ForEach
Parallel.ForEach(_streamingData.GetConsumingEnumerable(),
item => Process(item));
// PLINQ
var q = from item in _streamingData.GetConsumingEnumerable().AsParallel()
...
select item;

Stephen then mentions

"when passing the result of calling GetConsumingEnumerable as the data source to Parallel.ForEach, the threads used by the loop have the potential to block when the collection becomes empty. And a blocked thread may not be released by Parallel.ForEach back to the ThreadPool for retirement or other uses. As such, with the code as shown above, if there are any periods of time where the collection is empty, the thread count in the process may steadily grow;"

I do not understand why the thread count would grow?

If the collection is empty then wouldn't the blockingcollection not request any further threads?

Hence you do not need to do WithDegreeOfParallelism to limit the number of threads used on the BlockingCollection

usr · Accepted Answer · 2012-01-29T10:18:41.600

The thread pool has a hill climbing algorithm that it uses to estimate the appropriate number of threads. As long as adding threads increases throughput, the thread pool will create more threads. It will assume that some blocking or IO happens and try to saturate the CPU by going over the count of processors in the system.

That is why doing IO and blocking stuff on thread pool threads can be dangerous.

Here is a fully working example of said behavior:

        BlockingCollection<string> _streamingData = new BlockingCollection<string>();

        Task.Factory.StartNew(() =>
            {
                for (int i = 0; i < 100; i++)
                {
                    _streamingData.Add(i.ToString());
                    Thread.Sleep(100);
                }
            });

        new Thread(() =>
            {
                while (true)
                {
                    Thread.Sleep(1000);
                    Console.WriteLine("Thread count: " + Process.GetCurrentProcess().Threads.Count);
                }
            }).Start();

        Parallel.ForEach(_streamingData.GetConsumingEnumerable(), item =>
            {
            });

I do not know why the thread count keeps climbing although it does not increase throughput. According to the model that I explained it would not grow. But I do not know if my model is actually correct.

Maybe the thread-pool has an additional heuristic that makes it spawn threads if it sees no progress at all (measured in tasks completed per second). That would make sense because that would likely prevent a lot of deadlocks in applications. Deadlocks can happen if important tasks cannot run because they are waiting for existing tasks to exit and make threads available. This is a well-known problem with the thread pool.

Example: Modern SSDs have multiple internal IO path. Mine has 4. When you have 2 cores you still need 4 threads to saturate your SSD. The thread pool algorithm would actually try to create 4 threads (at least if it was working correctly). It would spawn threads until 5 have been created. Then, it would notice that the 5th did not increase throughput, so it would retire the 5th thread. — usr, Jan 28 '12 at 19:58
The details of this process are not documented. But I have seen it in practice frequently when my tasks did IO. — usr, Jan 28 '12 at 19:59
Sorry I'm confused if the thread is not improving throughput(like being blocked), then why is this an issue in Stephen Toub's example? is the only difference here is that the blocking thread cannot retire? But still why would that increase the thread count? — TheWommies, Jan 29 '12 at 00:05
I added a lot of explanation. I am quite convinced that my guess is correct. — usr, Jan 29 '12 at 10:19
@usr The additional heuristic you were correctly guessing at is the "starvation avoidance" that's described in the quote in [this answer](http://stackoverflow.com/a/3488869/292555) — Omer Raviv, May 17 '12 at 21:19

Streaming Data BlockingCollection

1 Answers1