0

I am trying to improve the performance of a Drupal (6) installation under load. One web server, one db server (MySQL5) both are Single Socket Quad Core Intel Xeon 2.5GHz with 16GB RAM. On the web server, I am running APC to improve PHP's performance. Running a memcached server with 6 bins on the DB server. Using Apache Solr to power search, Solr lives on the DB server as well.

The bottleneck appears to be Apache on the web server, as it runs out of memory under the load. No single process appears to be hogging too much of the available system resources.

Any input would be greatly appreciated. If there are questions I can answer about the setup, please ask away.

Thank you!

rrlange
  • 1
  • 1

4 Answers4

3

Since we're talking PHP I assume you're using the Apache prefork MPM. Given Rasmus Lerdorf's attitudes on multithreading, you should continue to use Apache prefork or a similar non-threaded setup with another HTTP server (such as nginx).

4 quick wins come to mind. I can't say from OPs original question if this has already been done -- if not, these 4 improvements should bring a substantial improvement.

a) Move static file serving (CSS, images, etc) away from your PHP Apache instances. With the Prefork MPM, when you're sending a single image to a user, a whole Apache child with the full PHP runtime is kept occupied to send that single image. This could be a 20-40 MB Apache process occupied for 0.1 to 3 seconds while the user downloads that image -- in the same time that Apache instance could have served tens or hundreds of dynamic pages. Suggestion: Set up a dedicated server for static resources, or use a cheap CDN like Amazon CloudFront.

b) Turn off HTTP keepalives to the Apache instance that serves dynamic (Drupal) content. HTTP keepalive is great and has its uses, but again, with keepalives on & Apache prefork MPM a single end user browser will keep 'bloking' (i.e. occupying) an Apache child for lengthy periods.

c) Verify that you're sending the proper HTTP headers, especially that static content such as images, CSS etc is cache-able. Otherwise the user's browser will re-downlod this static content on each page view, which is a total waste. Use Yahoo! YSlow, Mark Nottingham's RED or similar tools.

d) Set up another web server, and just use plain DNS round robin to distribute load between the two web servers. I assume Drupal can keep session state stored on the MySQL database (most PHP apps can)? If so, then the session store is taken care of, and you can use DNS round robin.

DNS round robin is not ideal. It does not handle a failure of a server, it doesn't guarantee perfect load distribution. But it's easy to set up, and usually works well enough for a simple setup with 2 or maybe 3 web-servers.

  • Jesper - thank you so much for your suggestions. Lots of really, really helpful hints here. A couple of comments: a) I will try this today. It means patching Drupal core to update file paths, but this might really lighten the load on Apache. c) I thought I had this done correctly, turns out that ETags are still set (need to fix) and ExpiresByType for images set to"access plus 1 hour" - I really need to up that. d) I will do this, if all else fails. Thank you again for the suggestion. b) Provided I can a) and c), are there any major drawbacks to turning off HTTP keepalives? Thanks! – rrlange Sep 27 '09 at 12:27
  • @rrlange: Turning off HTTP keepalive should more or less always be problem-free, regardless of your settings for "a)" and "c)". As always, make backups first and measure impact after the change, but it should be fine. You should see a marginally longer latency for browser connections, due to the time it takes to complete the 3-way TCP/IP handshake, but just in the order of max 150ms or so (depending on link RTT). This should be easily overshadowed by the extra scalability from freed up CPU and RAM on the server. All assuming you're not already using Perlbal or similar HTTP multiplexer. –  Sep 27 '09 at 20:29
  • @rrlange: Regarding "a) Move static file serving", I would myself not move all images away from the Drupal Apache instances, unless that's somehow super-simple to do with Drupal. I'd move very often requested resources, such as site-wide CSS, Javascript, logo images etc away. For other images used inside an article etc the benefits of centrally managing these images in a CMS are probably more important than server-side performance? –  Sep 27 '09 at 20:35
  • @Jesper Mortensen: Thank you again for your comments. - We have a way of using PUSH to load all static resources (.js, .css, .png etc.) to a CDN. Files will still be managed locally, just referenced with the CDN URL in the final output. Local resources are still available in case of CDN failure (not likely). – rrlange Sep 28 '09 at 13:59
0

You can setup nginx or lighttpd to serve static files (css/js/images/videos) in front of the apache server.

Another technique is static page caching. I think the Boost Module can do something like this. Or if you have the time and the resources create your own drupal modules to generate static html files instead of the php dynamic pages.

hdanniel
  • 4,293
  • 23
  • 25
  • HD - I have tried Boost, and it has helped some...but it I am reserving that as the last resort solution for highly-trafficked pages (homepage etc.) - thank you for your comment and suggestion. – rrlange Sep 27 '09 at 12:22
0

Have you tried pressflow it is Drupal with some performance and scalability tweaks.

Jeremy French
  • 675
  • 3
  • 12
  • 25
-1

Ditch Apache in favour of something lighter like nginx?

Adam Gibbins
  • 7,317
  • 2
  • 29
  • 42
  • Adam - thank you for your suggestion, I think I would want to try moving nginx in front of Apache first (to handle css, js & images). – rrlange Sep 27 '09 at 12:29