33

I've got some trouble with .NET's ThreadPool (.NET 4).

I've read that by default .NET has a limit of 25 threads per processor, but according to forum posts on SO and on other places, I can increase the limit with the below code.

void SetThreads(int threads)
{
    ThreadPool.SetMaxThreads(threads, threads);
    ThreadPool.SetMinThreads(threads, threads);
}

However, when I set the above to some arbitrarily high number, for example, 2000, and queue ~1000 items, I still only have ~33 threads running (.NET CLR takes ~5 threads), and ThreadPool.GetAvailableThreads() returns 1971 threads remaining.

Why doesn't the code above work?

foxy
  • 7,599
  • 2
  • 30
  • 34
  • 1
    How are you queueing up tasks on the thread pool? Showing your test code could really help. – Paul Turner May 14 '11 at 09:09
  • Your problem is that you should be running _less_ than 33 threads, not more. – H H May 14 '11 at 10:00
  • @Henk Holterman, it's not a large app - it's a quick wack-job HTTP load generator I made to try and load a local web server with a bunch of requests. Unfortunately it didn't turn out so well. – foxy May 14 '11 at 11:11
  • For a load-generator use Tasks and/or APM. – H H May 14 '11 at 11:14
  • Doesn't Tasks use the ThreadPool internally? I've never used APM, so I'll have a look at that. Just did some research, and it looks like APM also uses the ThreadPool? – foxy May 14 '11 at 13:38
  • The APM uses IO ports (look at the 2nd parameter in your methods). The TPL has some support for APM. And yes, it's all ThreadPool, what matters is _how_ you use it. – H H May 15 '11 at 08:10

3 Answers3

99

Firstly, your "knowledge" of the defaults is incorrect. The limit of 25 threads per processor was back from .NET 1.1. It was increased in .NET 2, and now:

Beginning with the .NET Framework version 4, the default size of the thread pool for a process depends on several factors, such as the size of the virtual address space. A process can call the GetMaxThreads method to determine the number of threads.

However, there's something else at play: the thread pool doesn't immediately create new threads in all situations. In order to cope with bursts of small tasks, it limits how quickly it creates new threads. IIRC, it will create one thread every 0.5 seconds if there are outstanding tasks, up to the maximum number of threads. I can't immediately see that figure documented though, so it may well change. I strongly suspect that's what you're seeing though. Try queuing a lot of items and then monitor the number of threads over time.

Jon Skeet
  • 1,421,763
  • 867
  • 9,128
  • 9,194
  • 1
    This should be the accepted answer, it clearly explains what freedompeace is experiencing – Fayilt Sep 05 '13 at 12:17
  • 1
    That was the answer! I noticed that strange half second, sometimes one second where it did nothing at all. I have waiting Tasks and other Tasks that wait to be started in consequence, and everything is slow. I need about 10 threads for my work. Setting ThreadPool.SetMinThreads to like 20 (default was 8) worker threads fixed my issue immediately! Since I need those threads later as well, I won't decrease the minimum again. – ygoe Feb 11 '14 at 22:20
8

From the MSDN :

When demand is low, the actual number of thread pool threads can fall below the minimum values.

Read this too: Patterns for Parallel Programming: Understanding and Applying Parallel Patterns with the .NET Framework 4

Liam
  • 27,717
  • 28
  • 128
  • 190
Emond
  • 50,210
  • 11
  • 84
  • 115
  • 1
    I queued ~50k tasks and set the ThreadPool's min + max threads to 20k, yet only ~25 threads were used. Is this expected? – foxy May 14 '11 at 11:09
  • Did any/the tasks complete or get in a wait state? It is up to the manager to create threads. On some machines it might be better to not have a lot of threads because the threads will suffer from thread starvation. – Emond May 14 '11 at 11:21
3

Firstly check this link, especially this remark:

If the common language runtime is hosted, for example by Internet Information Services (IIS) or SQL Server, the host can limit or prevent changes to the thread pool size.

Then you should check the return value of ThreadPool.SetMaxThreads(threads, threads) method. Maybe it returns false?

Liam
  • 27,717
  • 28
  • 128
  • 190
Grook
  • 395
  • 1
  • 6
  • 19