2

I'm stress testing my UDP proxy on a machine with 8 logical CPUs. At around 14 thousand UDP clients, one of the CPUs (CPU 4) starts spiking and UDP throughput (as shown by Task Manager) plummets to zero. I used WPR to record CPU usage during the spike and WPA shows CPU 4 handling twice as many DPC/ISR fragments (~11K) as all the other CPUs (~5K each). The longest DPC fragment ran for 232 milliseconds, the next 5 DPC fragments all ran for over 100 milliseconds, and the next 57 ran between 1 and 90 milliseconds.

A fragment is defined as "a period of time during which a DPC or ISR ran uninterrupted." (see https://msdn.microsoft.com/en-us/library/windows/hardware/jj679884.aspx)

Microsoft recommends that DPC fragments run for no longer than 100 microseconds! (see https://msdn.microsoft.com/en-us/library/windows/hardware/ff546551(v=vs.85).aspx)

bxvbda.sys is the module (Broadcom NetXtreme II GigE VBD) and um_bdrv_dpc is the function for all of these long running DPC fragments.

So my question is: Am I interpreting the WPA data for DPCs correctly?

All CPUs DPC/ISR

Marc Sherman
  • 2,303
  • 14
  • 22
  • 2
    I think you are interpreting the data correctly. If you shared the trace then it would be easier for experts to say for sure. CPU Usage (sampled) should show what the driver is doing during those long DPCs. Ultimately you'll need to talk to Broadcom. Give them a trace and tell them to fix their driver. – Bruce Dawson Jul 13 '16 at 03:10

1 Answers1

3

I've been analyzing etl traces for sometime and yes you appear to be looking at this correctly. You don't mention the version of server you are using, but UDP traffic on some versions of Windows Server need a hotfix. That might be worth investigating.

The below post is pretty awesome post on analyzing DPC issues, might want to take a look at it as well. Essentially you walk through the trace and look for over 100ms. Now in Windows 8 this changed a lot though, so what OS (again) :)

http://blogs.technet.com/b/craigf/archive/2014/02/03/a-backup-server-flooded-by-dpcs.aspx

In Windows 8 you can see we changed how DPCs are calculated, reference below as well.

Key points:

"Why is DPC Latency so bad in Windows 8? I used a program DPC Latency Checker to measure. Windows 7 shows about 90u while sitting on the desktop but Windows 8 CP never shows anything lower than 1000u."

"On Windows 8, it reports DPC latency of 1000us (1ms) at the very lowest. It never reaches below 1000us, ever. It regularly spikes up in the 2-3ms (2,000-3,000us) range, and often spikes into the 20-30ms range during processing."

http://answers.microsoft.com/en-us/windows/forum/windows_8-networking/why-is-dpc-latency-so-bad-in-windows-8/05ef48a6-7775-4526-9e5b-32c5951b8dbf

Jeff Stokes
  • 46
  • 6