0

I'm debugging a system hang/freeze issue by forcing a complete memory dump (ctrl + scrl + scrl method) and I don't understand some of the data.

When I run !exqueue 6 I see 6 Critical, 8 Delayed and 1 HyperCritical thread but each contains a similar stack with only the following calls:

nt!KiSwapContext+0x7a
nt!KiCommitThreadWait+0x1d2
nt!KeRemoveQueueEx+0x323
nt!ExpWorkerThread+0xe9
nt!PspSystemThreadStartup+0x5a
nt!KxStartSystemThread+0x16

To my knowledge these are all threads that were created but not given any work, right?

Is this what is actually happening on the system at the time of the dump or is this just the effects of forcing a dump with this method?

Is this also why the only running threads are actually intelppm under the Idle PID and the breakpoint?

                        [fffff80003617180 Idle]
   0.000000  fffff80003616cc0 ffff8835 RUNNING    nt!KeBugCheckEx
   0.000000  fffff880009f9fc0 ffff92bb RUNNING    intelppm!MWaitIdle+0x19
   0.000000  fffff88002f6ffc0 ffff9191 RUNNING    intelppm!MWaitIdle+0x19
   0.000000  fffff88002fe1fc0 ffff93c4 RUNNING    intelppm!MWaitIdle+0x19

It doesn't seem like I'm actually getting an accurate picture of the work queue or running threads at the time of the dump. Am I misinterpreting the data or is it all because of how it was captured?

Any help would be appreciated.

mitchellJ
  • 734
  • 3
  • 9
  • 32
  • 1
    !exqueue shows threads that are in wait state in the worker thread queue it will be problem only if current is > maximum(old windbg's display) or concurrency ratio is > 1 (newer windbg's display) all the waiting threads will probably have the same stack – blabb Mar 10 '16 at 08:11
  • Awesome, thanks. So the essentially the system work queue looks like it genuinely didn't have anything waiting which would indicate that other threads are running smoothly - right? – mitchellJ Mar 10 '16 at 14:56
  • 1
    it means the problem doesn't appear to be in the worker thread queue some thread that is not a worker thread may not be running smoothly and or deadlocking there are several commands like !locks etc that you can employ to investigate – blabb Mar 10 '16 at 15:53
  • try the **!running** command – magicandre1981 Mar 10 '16 at 16:50

1 Answers1

1

During boot your system will automatically create a number of system threads. Later on the system can create more system threads if it deems it necessary based on load. The system threads that you are showing does not have anything to do right now but they might have been busy earlier on. I don't think (6, 8, 1) threads indicates anything abnormal wrt. load.

The 3 threads that were executing intelppm were idling and then most likely power collapsed.

There could be other reasons for a system to become unresponsive than high load. Sometimes all it takes is one thread being deadlocked. So perhaps look at what other threads were doing and if any one of them have a "suspicious" looking call stack. Also did core 0 (which executed the bugcheck) do anything special before executing the bugcheck?

Rhansen
  • 26
  • 3
  • That's one of the items that makes me suspicious that the keyboard dump is muddying the waters. Showing the stack for ~0 only contains the thread that started and ended with the keyboard dump code. !ready shows nothing and !running shows just those idle threads (which I also find suspicious). Obviously something else was running on 0 when ctrl-scrl-scrl was hit and that thread was low enough IRQL that it blocked it. But how is that traced? – mitchellJ Mar 10 '16 at 14:54
  • So the stack for core 0 started at "nt!KiIdleLoop" and ended up in "bugcheck"? I don't think the keyboard crashdump muddys the water in any way. There is an interrupt and USB driver queues a DPC which ends up calling bugcheck. It sounds like the thread on core 0 was also idling! – Rhansen Mar 10 '16 at 19:48
  • So the interrupt is supposed to happen on the thread running at that time rather than creating a new thread that blocks what was running? Thus if something is actually running the stack for that thread should be interrupted by the USBPORT Xdpc_Worker? – mitchellJ Mar 11 '16 at 03:55