-1

Got an issue on Windows OS - NP pool slowly increases in size, and after several weeks consumes up to several GBs. Tried to determine the faulty driver(?), but running this:

xperf -on proc_thread+loader+pool -stackwalk poolalloc -MaxFile 1024 -FileMode Circular

twice (after machine restart and after several hours) gives me this:

WPA analysis

i.e., I do not see any information about processes\dlls which can be responsible for the leakage. Am I missing some arguments while taking the trace, or WPA is not enough in order to find some types of leakages?

@magicandre1981 The thing is that in my case Stack is not listed: WPA I can't rely on Impacting size here, because the leakage goes very slowly here, but 400 MB for NP pool is too big.

PoolMon shows that mainly Irp & FMic are leaking:

PoolMon right after restart

PoolMon after several hours

Kerberrus
  • 1
  • 1
  • I [answered it on superuser.com](https://superuser.com/a/949246/174557) some time ago. Put the columns in my shown order. In my answer I also showed that Razor Cortex. caused FMic and Irp usage. maybe you also use this software – magicandre1981 Jan 06 '21 at 14:11
  • @magicandre1981 added PoolMon screeshot in the initial message + WPA screenshot. The thing is that Stack is not visible here. – Kerberrus Jan 06 '21 at 16:51
  • add Type column and only care about AIFO (allocatied inside trace but not freed inside trace) – magicandre1981 Jan 06 '21 at 17:50
  • have you changed the column order as I told you? Do you see the cause now? If you need help, provide the zipped ETL file (onedrive, dropbox share links) – magicandre1981 Jan 08 '21 at 10:58
  • I apologize, didn't have time to check that. Will definitely update you. – Kerberrus Jan 10 '21 at 11:07
  • OK, I sorted the trace this way - https://imgur.com/a/1evxCo0 . Thre tag is only filled with OS modules. As I understood, anything inside Fmic\Even tag here can be a root cause, thus I simply need to check it one-by one, and there is no need to look at "impacting size" . A sidenote: tried to collect a trace for 2 hours, resulted in 20 GB file ( in order to notice the increase in "Impacting size". Unfortunately, while trying to sort NP pool by Type, WPA crashes :(. – Kerberrus Jan 10 '21 at 13:35
  • ok, FMic comes likely from Veeam. look for an update or remove it – magicandre1981 Jan 10 '21 at 13:48
  • Thank you very much, @magicandre1981!! So, in general - we simply need to check what modules are running under AIFO type during the trace; look for any suspicious modules and check them, am I right? Would long-lasting trace help us here, i.e. we could see that "Impacting size"\"size" is increased explicitly for this Tag\Module? Just interesting, how we would trace such leakage in ntdll or sqldk.dll files, for example.. – Kerberrus Jan 10 '21 at 13:55
  • yes, check the stacks that you see in poolmon for 3rd party dlls. also expand the IRP stack and look what you see there – magicandre1981 Jan 10 '21 at 13:57
  • so after 2 days is the issue gone after removing Veeam? – magicandre1981 Jan 12 '21 at 17:38
  • Upgraded to the latest patch - the issue is gone. Thanks again! – Kerberrus Jan 13 '21 at 19:24
  • ok, nice to hear this. Now you learned how to debug this. – magicandre1981 Jan 13 '21 at 20:21

1 Answers1

0

Big thanks to @magicandre1981. Was discussed already https://superuser.com/a/949246/174557, but just wanted to add - we should concentrate on AIFO pool type and search for suspicious modules inside each pool tag (in my case I was interested in Fmic, Even & IRP). Once it's done - disable application\driver one by one and check the results. For some reason WPA sorts the data by Pool Tag, and Type column is not enabled by default. Eventually, it should look like this Windows Performance Analyzer NP pool trace

Kerberrus
  • 1
  • 1