In IIS running an ASP.net application, why would CPU usage slowly increase, sharply drop, then rise to a throttled level?

Question

See image here: total CPU usage %

This trend is easily repeatable by running automated load tests on the application. After the CPU use settles at the apparently throttled level (~30%), application response times are VERY slow.

Answer 1

Try monitoring other perf counters as well. My first guess is to check the % Time in GC.

Answer 2

Thanks for the answers. I made sure that Debug is set to "False". I looked at the memory and % time in GC counters. Right after the sharp CPU drop, memory use goes up for a few minutes, and then goes back down (it is garbage collected). Overall % time in GC is not high.

I found some additional information in the System event log. The sharp CPU drops correspond to IIS 1011 events:

A process serving application pool %1 suffered a fatal communication error with the WWW service. The process ID was %2. The data field contains the error number.

Answer 3

Looking at that, I have two culprits in mind:

• Objects ending up on the large object heap. This will cause performance problems with the web app, but not show up in the GC time counter. However, I'm leaning away from this as it would tend to be related to high memory use and out of memory exceptions/errors rather than cpu use.
• Something is not disposed correctly, likely because it's a global object they reference on every request. I'm thinking it's related to string concatentation, because that's the operation that's most often related to high cpu use. Put those clues together and my gut tells me to look for a logging system used with the app. This is an object that's going to be hit every request and deals with a lot of strings.

No matter what it is, I think what you need to do here is send the app back to the programmers. There's not likely a server config option you can set that will magically fix things.