Strategy for caching

Question

I have a 5 dedicated servers each running the same version of an application. I'm looking to massively step up our caching strategy to take some load off MySQL as this is our main bottleneck. All the sites are LAMP and each server has approximately 20 hosted sites. I'm planning to use Zend_Cache for running the caching and it has a number of adapters i can use - the main ones are Memcached / APC / File and SQLlite. I have a number of things that i'm looking to cache but they mostly fall into the region of either large DB result sets or HTML snippets. For HTML snippets the file based solution would seem ideal but a number of people have mentioned this could introduce a sizeable IO overhead - especially if it was done across a large number of sites as this would be. I do also like the idea of Memcached because of its speed - i've had a vague idea to try and set up a dedicated Memcached server purely for handling caching and have all my other servers use this although i'm not sure how much of a good idea this is and how much of its speed would be lost by the RAM not actually being directly local to the machine making the request.

Ultimately it would be good to get some solid advice on how performant (best performing) the different adapters are, what they're best suited to, and what our best options are.

Thanks.

Answer 1

Sounds like you've already chosen a solution and are now just trying to sort out the details.

I have a 5 dedicated servers...LAMP

So you've already got cyclic master-master mysql replication running on all 5 hosts, HTTP content served up with appropiate caching instructions via reverse proxies in a shared cluster?

large DB result sets or HTML snippets

And you've optimized the schema around the large result sets and are using something like ESI caching for the HTTP snippets.

i've had a vague idea to try and set up a dedicated Memcached server purely for handling caching

Leaving aside the fact that this would introduce a single point of failure, how is this more efficient than a single DBMS server with lots of result set caching?

how much of its speed would be lost by the RAM not actually being directly local

Why don't you measure it and find out?

Any memory based caching solution is not going to yield much (any?) beneift compared with the mysql result cache - and if you are trying to cache large datasets then you're quickly going to flush a small memory cache.

Sit down with a spreadsheet and start doing some modelling to get estimates of how many and how large the cacheable entities will be and how frequently they might be reused given an infinitely large cache.