0

I have a server running CentOS 5, MySQL 5.1, and PHP 5.2. I have a problem that I'm not sure where to start in troubleshooting. It seems that during periods of high traffic, Apache will occasionally have a delay (or timeout) in retrieving a requested web page. Basically, I visit a page and it just hangs on fetching it.

The problem is though that I can visit 100 pages afterward with no problem, so it's very difficult to repeat. I think the cause is improper configuration that does not allow for enough simultaneous connections to either Apache or to MySQL.

Could you please give me some tips on things I can test out to try and rule some of the possible causes out?

Dave Johnshon
  • 3
  • 2

2 Answers2

0

take a peek at this block in your httpd.conf 

<IfModule prefork.c>
        StartServers 2
        MinSpareServers 4
        MaxSpareServers 8
        ServerLimit 75
        MaxClients 75
        MaxRequestsPerChild  1000
</IfModule>

then during one of these peak periods do a:

ps ax | grep httpd | wc -l

should give you the number of apache processes running - if it is close to your server limit directive - bump it in your httpd.conf. your 'lag' sounds like the time it takes apache to wait for another process to timeout/die and spark up a new one.

-sean

Sean Kimball
  • 869
  • 1
  • 8
  • 24
  • Thanks. So if I keep an eye on the number of httpd instances running throughout the day, even if I can't reproduce the lag it would probably be a good indicator of an issue if I am say 85% of my max all the time? – Dave Johnshon Aug 08 '11 at 16:12
  • hmmm if 85% is the norm, then you really have no room for any kind of spike. Have you got the physical resources to increase your ServerLimit/MaxClients? Tuning those directives in that little block may help - take a look at the MaxClients in the apahe docs - http://httpd.apache.org/docs/2.2/mod/mpm_common.html#maxclients we can't REALLY tell you what the correct number should be, but you need to balance your traffic load with your memory available to apache – Sean Kimball Aug 08 '11 at 16:37
0

A few things you can look at:

  • Examine the output from top during both low and high load times. View it during low load to get an idea of things when everything is running fine and compare them to the high load stats. Look at things like: memory consumption, CPU usage, IO usage, which processes are using memory/CPU, are there any processes in "uninterruptible sleep" (a 'D' in the stats column).
  • Use netstat to see if your getting a high number of incoming requests during the high load times. Look for a large number of connections with the same IP.
  • free, vmstat and iostat can give you more information on the memory and IO. In particular make sure you are not using any virtual memory.
  • Enable Apache's server-status and check what the server is doing during the high load times. This can also tell you if a single client is misbehaving (look for multiple requests with the same IP).
  • If possible you can try using a benchmarking program like ApacheBench (ab) to test your site and try to duplicate the issue.
  • Check the relevant log files in /var/log/ including Apache, MySQL, system logs, etc... for any obvious error or warning messages.
uesp
  • 3,414
  • 1
  • 18
  • 16
  • For the netstat one, what would be considered a large number of requests from 1 ip? is 10 too many or are we talking more like 50-100? – Dave Johnshon Aug 08 '11 at 21:19
  • It does depend on your specific application/site but in my case I rarely see more than 10 concurrent ESTABLISHED connections from each client and typically only 1-3. If I ever see 20 or more at one time or a client continually using a dozen it is likely something "abnormal". To view only the current connections run a command like `netstat -an | grep ESTABLISHED` otherwise you will get a large number of WAIT/SYNC connections which can be useful but harder to determine what is good/bad. – uesp Aug 09 '11 at 13:11