2

I recently launched a web app that gets constantly pegged at 100% CPU. My server is a 512 MB Linode with nginx sitting in front of Apache (MPM Worker) serving a Django app. This app is unusual in that it has to do heavy database processing < 5% of the time (~2-5k queries/request). I only have a small number of users currently (50 in a day, ~5-10 at any given time). Now I understand doing several thousand queries is over the top and I'm looking into optimizing it, but I don't understand why the Apache processes go to 100% and stay there even though no one is currently visiting the heavy load page.

Here is part of my Apache config:

<IfModule mpm_worker_module>
 StartServers          2
 MinSpareThreads      25
 MaxSpareThreads      75 
 ThreadLimit          64
 ThreadsPerChild      25
 MaxClients          150
 MaxRequestsPerChild   0
</IfModule>

Here's my the top few lines from 'top':

10257 www-data  20   0  370m 143m 4052 S  100 29.0  16:19.47 apache2                                                     
1 root      20   0  2728  224   24 S    0  0.0   0:01.34 init                                                        
2 root      20   0     0    0    0 S    0  0.0   0:00.00 kthreadd

Any help would be much appreciated!

Rick-777
  • 103
  • 3
mathew
  • 213
  • 2
  • 7
  • I'm not sure but would there be a reason that some of the requests keep being done when someone quits in the mid of a request? It might be that your script keeps running, which results in high load. However it is my understanding that apache stops executing a script when a request is dropped. – Lucas Kauffman Apr 21 '12 at 22:13
  • It didn't look like Apache would ever stop the request, which was interesting. I would like to dig deeper and find out why. – mathew Apr 22 '12 at 18:27

1 Answers1

0

This turned out to not be a server problem. There was a particular codepath that was very CPU-intensive (it was O(n^2) operation in the worst-case). I tracked down the problem by getting in touch with a particular beta user who was having the problem and copying their data to my staging server and testing it. I was able to reproduce it and fix it from there.

I would still like to know why the Apache request would continue to process the request when the user had stopped their side of the connection though.

mathew
  • 213
  • 2
  • 7