4

We have an application where a low response time is crucial. Everything is working fine except for a few time per week where 1 random call takes around 5000ms to complete vs 200ms for other calls. A call is made a few seconds before so it is not related to the dlls being unloaded. The time is random, there is no App pool recycling involved.

IIS Time taken : 5000ms

ASP.NET\Request Execution Time : 5000ms

Our counters in the code : 200ms (Log at the beginning and at the en of execution)

It seems the call is stuck somewhere before our code is hit. We monitored the perf counters and when this happens the call is not queued and there is no wait time.

IIS config has been optimized with custom processModel values (as described in http://support.microsoft.com/kb/821268) for better throughput.

We monitored the disks, network, memory and CPU and everything looks okay.

What could cause such a spike (IIS thread allocation maybe?) What other counter beside the obvious ASP.NET, .NET CLR, System, Thread, could help us?

Config : Windows 2003 SP2 (hosted on VMWare ESX)

IIS 6

.NET Framework 3.5

Bobby Hache
  • 41
  • 4
  • It sounds silly but I've had random issues with VMs that didn't occur on comparable physical hardware. Is it limited to one machine? Can you try on real HW? – Kieren Johnstone Oct 13 '11 at 19:30
  • No, it happens on all VMs. We don't have access to real hardware but I will try to invistigate this path anyway. – Bobby Hache Oct 13 '11 at 19:42
  • Ok. I'm not an 'old man' but sometimes I truly believe VMs are overrated. Every big commercial situation I've seen crazy shit clear itself up on a real machine. – Kieren Johnstone Oct 13 '11 at 19:50

2 Answers2

0

I would start with this article WCF Request Throttling and Server Scalability

In particular the Throttling Tuning section:

In order to achieve high throughput on high-end servers, you would need to make the following throttling tuning:

  • Increase ASP.NET thread throttling to allow more concurrent worker threads to handle requests
  • Increase WCF service throttling to allow more concurrent requests
ErnieL
  • 5,773
  • 1
  • 23
  • 27
  • Great article, thank you. But we already have performed those configs at the WCF level. I will look into ASP.Net level though, I'm not sure we did it. – Bobby Hache Oct 13 '11 at 19:38
  • Also, when this happens, there is no other call. I would be surprised that there would be throttling involved – Bobby Hache Oct 13 '11 at 19:50
  • In that case I would stop looking at your service and focus on what the WCF channel depends on before your code is called. For example, what are you using for security? Is there an Active Directory or STS round trip happening before your code executes? Something is expiring after 2-3 days, timing out after 5 sec, being refreshed, and then working... My honest opinion is if this lasts much longer, you should look at upgrading the OS to server 2008 R2. – ErnieL Oct 14 '11 at 03:00
0

Have you tried tracing the service and the client to find out if there are secutrioty negotiations or stuff like that slowing you down?

See: SVC Trace viewer(msdn)

Mikael Eliasson
  • 5,157
  • 23
  • 27
  • It will be hard to use tracing since it is in production and it happens once in 2-3 days. Trace files would be huge... – Bobby Hache Oct 13 '11 at 20:19