16

I have generated dumps on four servers and am analyzing the output of !threadpool and !threads. I noticed the roughly consistent following output:

0:024> !threadpool
CPU utilization 0%
Worker Thread: Total: 2 Running: 0 Idle: 2 MaxLimit: 200 MinLimit: 2
Work Request in Queue: 0
Number of Timers: 27
Completion Port Thread:Total: 2 Free: 0 MaxFree: 4 CurrentLimit: 2 MaxLimit: 200 MinLimit: 2


!threads -special
ThreadCount: 32
UnstartedThread: 0
BackgroundThread: 19
PendingThread: 0
DeadThread: 13
Hosted Runtime: no

My questions are:
1)How can I determine what the source of those 27 timers are?
2)What does 13 dead threads mean?
3)One of my threads is marked as having a lock. If i switch to that thread and run !clrstack, i see the following - is it related to my timers?


0:027> !clrstack
OS Thread Id: 0x14cc (27)
*** WARNING: Unable to verify checksum for System.ni.dll
Child-SP         RetAddr          Call Site
000000000ca6e1a0 000007fef5c06477 System.Threading.WaitHandle.WaitAny(System.Threading.WaitHandle[], Int32, Boolean)
000000000ca6e200 000007fef673175b System.Net.TimerThread.ThreadProc()
000000000ca6e2d0 000007fef67c95fd System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
000000000ca6e320 000007fef84d1552 System.Threading.ThreadHelper.ThreadStart()
JohnW
  • 2,982
  • 1
  • 28
  • 30

1 Answers1

19

1)How can I determine what the source of those 27 timers are?

Try to find instances of TimerCallback (for Threading.Timer):

!dumpheap -type TimerCallback

Then dump the callback properties (where callback address is the "Address" from the dumpheap output):

!do <callback address>

Then dump the Value address of the _target property:

!do <_target address>

That should spit out the object that holds a reference to the TimerCallback, which should lead you to where the timer was created.

I recommend checking out Tess Ferrandez's debugging labs, if you haven't already.

2)What does 13 dead threads mean?

My understanding is that a dead thread refers to a C++ thread which no longer has an active OS thread, but still has references and thus cannot be destroyed (C++ threads use ref counting).

A C# thread holds a reference to a C++ thread, and if your managed code keeps a reference to a C# thread, then that could be your problem.

This post on Yun Jin's blog might be of some interest to you. (wayback archive)

3)One of my threads is marked as having a lock. If i switch to that thread and run !clrstack, i see the following - is it related to my timers?

That looks like a (System.Threading) timer thread waiting for its interval to elapse.

juFo
  • 17,849
  • 10
  • 105
  • 142
Richard Szalay
  • 83,269
  • 19
  • 178
  • 237
  • 1) Thank you. SUrprisingly, I was provided with 80 instances when I did a dumpheap. The values make sense for the app we have, but I wish there way a way to correlate these to the 27 listed by !threadpool. 2)Thanks for the link to the blog. It looks relevant. – JohnW Jan 12 '10 at 17:59
  • Is there an updated post on how to debug DeadThreads in CLR 4? Yun Jin's blog post cited above is for mscor 2 (I think) and does not work in CLR 4 because the symbols do not exist in CLR 4. – Kevin S. Miller Aug 25 '21 at 15:40