0

I have a server running very slowly due to high CPU usage. I think this must be a configuration issue but I can't put my finger on why. The server is running Plesk and I've configured PHP to run as a Fast CGi application.

Server spec:

Quad-Core AMD 2.2GHz
4GB RAM
Ubuntu 10.04
Apache 2.2.14
PHP 5.3.2

Websites:

1 x Drupal 7
2 x WordPress 3.x

Apache config:

KeepAlive On
Timeout 30
MaxKeepAliveRequests 0
KeepAliveTimeout 2

<IfModule mpm_prefork_module>
    StartServers          4
    MinSpareServers       5
    MaxSpareServers       10
    MaxClients            20
    MaxRequestsPerChild   5000
</IfModule>

Notice the high CPU

I have APC enabled plus the Drupal site has all caching turned on, including caching in views. I have another server running over 100 sites on CentOS and cpu usage is very low. Any ideas what could be causing this?

Heres a mod_status output screenshot

enter image description here

Also there is no output in the apache error.log apart from MaxClients being exceeded.

top CPU and Memory lines added 25th Apr enter image description here

Frank Martin
  • 101
  • 1
  • 3
  • 1
    How much available RAM do you have? ( `free -h` ) , each php5-cgi process is consuming about 100 MB (which seems 3x on the high side) so try: a) increasing MaxClients according to available RAM b) unload unneeded modules c) if still high CPU, reduce TimeOut, take out KeepAlive – LinuxDevOps Apr 22 '14 at 16:02
  • Are you sure you mean prefork, and not mpm_worker with mod_fcgid? I believe if you use prefork, you shouldn't see php5-cgi processes in your process list, but apache2-processes. If you visit the websites on the affected server, how do you rate the performance? Are they slow to respond, or just as fast as other websites? – Martijn Apr 22 '14 at 22:01
  • 4GB RAM (1.6GB Free). In plesk I've switched from fcgi to apache module several times to test things each time they are still affected by the prefork config. I've taken out keep alive and put max clients down to 10 which has taken CPU down to about 80% but sites are slow as hell – Frank Martin Apr 23 '14 at 10:51
  • What server MPM is listed when you set fcgi and run `apache2ctl -V`? And what if you use 'apache module'? – Martijn Apr 23 '14 at 13:57
  • server/mpm/prefork, In plesk you can set fcgi/apache per domain so one can be fcgi while another apache module, with all set to apache module apache2ctl -V is still prefork – Frank Martin Apr 23 '14 at 14:10
  • Ok, let's assume Plesk has this configuration stuff covered. Sites are slow even though cache is used.. Process list shows processes in running and sleeping status. Would be interesting to see how the CPU-usage is divided. This can be seen in `top` if you press `t` for task/CPU summaries. If `%wa` is high or has peaks, also check `iotop` to see if there's any problem with I/O. It's possible I/O is bad for some reason and this can cause all kind of issues. – Martijn Apr 23 '14 at 15:46
  • Ok, %wa was high, disk io (on New Relic) was showing very high 95%. iotop showed an access_log file using 90%, I realised access logs werent being rotated so some were over 10GB, configured and manually run logrotate which took io down considerably but cpu is still high. 80% ish – Frank Martin Apr 24 '14 at 14:08
  • Mh, I don't think large log files should slow down I/O just on it's own. I/O is usually a problem with the speed at which the log files are filled. As an experiment, could you try to stop logging to access_log and error_log for a while and see if this makes a big difference? And could you post the 4 CpuX-lines from `top` to see what how cpu% is divided now? – Martijn Apr 24 '14 at 16:47
  • I've added a screenshot above. – Frank Martin Apr 25 '14 at 08:28
  • Also disabled apache error/access logs temporarily, doesn't look like theres any difference maybe that %wa sometimes goes down to zero now. – Frank Martin Apr 25 '14 at 08:55

2 Answers2

1

No amount of Apache or PHP tuning will help you, it's time to see what makes Drupal or Wordpress so slow. I guess you have some custom modules in one or both of them and some custom module is dragging the performance down. Try disabling custom modules one by one and see if that helps. Also enable logging in your database and spot the slow and/or excessive queries.

When it comes to Drupal, Boost can be an excellent performance boost. With WordPress, W3TC tends to help a lot.

Janne Pikkarainen
  • 31,852
  • 4
  • 58
  • 81
  • You are right they do have a lot of customisations. I've disabled what I can but I guess this could be a lesson to teams that push deadlines so much that quality of code is affected, always has repercussions. – Frank Martin Apr 29 '14 at 08:34
0

My problem is now fixed, below is the steps I took.

  • Migrated server to Ubuntu 14 LTS with latest software including PHP 5.5
  • Split MySQL load on to its own server instance
  • Uninstalled Drupal modules that weren't needed

Note: None of the above fixed my CPU issue

Added Honeypot from http://www.projecthoneypot.org/ and BOOM, CPU load reduced to under 10%. So my issue was down to HTTP spam coming in at over 5 requests per second (400,000 per day) on a page with heavy SQL queries but one problem does remain... My site does NOT appear to scale well, I guess if this was legitimate traffic I'd have to look at optimising / caching wherever possible as Janne suggested.

Thanks to everyone for their advice.

Frank Martin
  • 101
  • 1
  • 3