1

We are having a very strange situation with one of our websites. On some days the performance is consistently fine across the entire day, on others days consistently slow across the entire day.
For example a small page ('get_status.asp') is requested about 10 times per second by the 3000 unique visitors per day. On good days it takes 30-40ms, on bad days 250-500ms.

The application pool is recycled every night at 3am.
I found out that on bad days IIS is running in single threaded mode executing a maximum of two requests at a time, on good days there seems to be no limit.

A DebugDiag report shows on bad days:

COM+ STA ThreadPool Report
Max STA Threads     50 
Min STA Threads     4 
Current STA Threads     4 
g_activitiesPerThread     1 
EmulateMTSBehavior     False 
STA Threads In-Call     0  (none)

Even though 50 threads are allowed, only 4 are used (2 processors). In perfmon I can see that a lot of ASP-requests are being queued, causing the slow response times (because of high Reques wait times).

On good days all 50 threads are used (see Current STA Threads):

COM+ STA ThreadPool Report
Max STA Threads     50 
Min STA Threads     4 
Current STA Threads     50 
g_activitiesPerThread     1 
EmulateMTSBehavior     False 
STA Threads In-Call     0  (none)

Some information

  • Server is Windows 2003 R2, on VM-Ware ESXI. 2 CPU's assigned, 2GB memory
  • Load is spread quite evenly across the day. At the busiest moment there are about 25 requests/second. The CPU's are running nicely at 25%, memory usage is 750MB (out of 2GB available)
  • The website only uses Classic ASP. No ASP.NET is involved.
  • Server Side ASP debugging seems to be able to cause this. However, it is turned off (I never used it, certainly not on a production server)
  • Storing COM-objects in Application or Session variables seems to be able to cause this. We do use Application and Session variables but only store basic types like strings and numbers.

What puzzles me most is the apparent randomness: I have seen Monday good, Tuesday bad, Wednesday good but also the other way around. Sometimes every day of the week is bad, sometimes an entire month(!) is good.

What makes IIS decide to use single threaded mode the entire day until the next application pool recycle?

Update

On another machine I have four websites running (for different URL's), each with their own application pool but all running the same software. Today I discovered that one out of those four is also stuck at exactly 4 thread, the others ones happy use up to 50 threads.

Jeff
  • 736
  • 5
  • 14
  • Does this shed any light - http://stackoverflow.com/questions/587420/iis7-only-serves-up-one-page-at-a-time-its-a-making-me-crazy – John Feb 10 '14 at 12:52
  • Unfortunately not, I read it before. That question was solved by disabling the serverside debugging which is already disabled. – Jeff Feb 10 '14 at 13:23
  • My first idea is to take a close look at the iis log, and focus on the period directly after the recycle. Is the response time directly slow? or is it degrading in the minutes/hours after the recycle? Compare the iis log for a good day and a bad day in detail. I know this isn't the solution but hey, I would have the influence of another asp page ruled out. My second idea would be to recycle at 0500 as well, the idea that it's related to the recycle was of-course introduced by your last sentence ;-) but I've been in situations I had to recycle twice (which is of-course uhhh beat me...) – Paul Feb 11 '14 at 17:31
  • Your puzzle: the apparent randomness must be your focus point, computers are just not good at random. – Paul Feb 11 '14 at 17:32
  • Today was a good day. I have set up a small scheduled task using Sysinternal's PSLIST.EXE to log the number of threads every 10 minutes. After the recycle the w3wp thread count is down to 28, it rises above 60 after about 6:30am when the first people start working. The maximum was 109 today. On a bad day w3wp never has more than 45 threads. – Jeff Feb 11 '14 at 18:39
  • @Paul, I will consider scheduling an IISRESET instead of (or next to) the application pool recycle. Maybe it is more robust. I might even consider a nightly reboot. – Jeff Feb 11 '14 at 18:46
  • Are you being limited by a SQL Connection pooling issue? I've seen that manifest itself in odd ways before. – Dave Mroz Feb 18 '14 at 19:27
  • Yes, we do use an ADODB connection with the Microsoft OLEDB Driver for Oracle to an Oracle 11 database (on another machine). By the active sessions in Oracle I can tell that it uses connection pooling. However I don't have a clue how that affects my thread count or how to prevent that. – Jeff Feb 18 '14 at 21:34

0 Answers0