0

On the customizable front page of our web site, we offer users the option of showing modules showing recently updated content, choosing from well over 100 modules.

All of the data is generated by MySQL queries, the results of which are cached via memcached. Our current system works like this: when a user load a page containing modules, module, they are immediately served the data from cache, and the query is added to a queue to be updated by a separate gearman process (so that the page load does not wait for the mysql query). That query is then run once every 15 minutes to refresh the data in cache. The queue of queries itself is periodically purged so that we do not continually refresh data that has not been requested recently.

The problem is what to do when the cache is empty, for some reason. This doesn't happen often, but when it does, the user is currently shown an empty module, and the data is refreshed in the gearman process so that a bit later, when the same (or a different) user reloads the page, there is data to show.

Our traffic is such that, if we were to try to run the query live for the user when the cache is empty, we would have a serious problem with stampeding--we'd be running the same (possibly slow) query many times as many users loaded the page. Is there any way to solve the "blank module" problem without opening up the risk of stampeding?

Karptonite
  • 1,490
  • 2
  • 14
  • 30

1 Answers1

2

This is an interesting implementation though varies a bit from the way most typically implement memcached in fronT of MySQL.

In most cases users will set things up to where queries are first evaluated at memcached to see if there is is an available entry. If so they server it from memcached and never query the database at all. If there is a cache miss, then the query is made against the database, the results added to memcached, and the information returned to the caller. This is how you would typically build up your cache for read queries.

In cases where data is being updated, the update would be made against the database, and then the appropriate data in memcached invalidated and/or updated. Similarly for inserts, you could either do nothing regarding the cache (and let the next read on that record populate the cache), or you could actively add the data related to the insert into the cache, depending on your application needs.

In this way you wouldn't need to take the extra step of calling the database to get authoritative data after getting initial data from memcached. The data in memcached would be a copy of the authoritative data which is just updated/invalidated upon updates/inserts.

Based on your comments, one thing you might want to try in order to prevent a number of of queries on your database in case of cache misses is to use a mutex of sorts. For example, when the first client hits memcached and gets a cache miss for that lookup, you could could insert a temporary value in memcached indicating that the data is pending, then make the query against the database, and the update the memcached data with the result.

On the client side, when you get a cache miss or a "pending" result, you could simply initiate a retry for the cache after a certain period of time (which you may want to increase exponentially). So perhaps first hey wait for 1 second, then try back gain in 2 seconds if they still get a "pending" results, then retry in 4 seconds, and so on.

This would amount in possibly more requests against the memcached server, but should resolve any problems on the database layer.

Mike Brant
  • 70,514
  • 10
  • 99
  • 103
  • Thanks for your reply. In fact for most queries we do use the more typical implementation, where we check memcache first, then query the database if the results are not present, caching the results. For queries that are on the front page, however, we would have stampedes if we did that. Unfortunately, we have enough different ways of returning data (sorting, filtering, etc) that regenerating the results whenever the data is invalidated is not currently an option, although it is on our long term "wish list". – Karptonite Aug 20 '12 at 17:03
  • OK That makes sense. I have updated my answer with a possible suggestions. – Mike Brant Aug 20 '12 at 17:18
  • Thank you--the idea of a "pending" result had occurred to me, but that sounds like a very good implementation. It has the downside of potentially delaying page loads to wait for caches to regenerate, but at least it doesn't slam the database. – Karptonite Aug 20 '12 at 18:03