How to cache a list of requests in Django views?

Question

I'd like to keep the list of 'Last Seen' 20 requests into a Django view. Also I want to avoid creating a separate model for it and just keep the requested urls in memcached, by pushing each new request to a fixed size queue and then retrieve the queue in views. But since cache is just key:value dictionary I'm wondering how best to achieve this?

Answer 1

One way to do this with memcache would be to store a pickled python object as a single key in the cache.

In this case we could use a Python deque which has exactly the properties we'd want for a list of 20 most recent items

Each time we record a new page view we need to update the deque, which means to get it, unpickle it, append, pickle it and set the new value back to memcache.

Fortunately Django's cache framework will handle the pickling and unpickling for us. However one thing we need to take care of is the possibility of a race condition - in other words if another process also updates the deque after we get our copy and before we have a chance to set it back to the cache.

For this reason we should use memcache's CAS ('compare-and-set') operation. There is an extended Django cache backend that enables CAS available here:
https://github.com/anentropic/django-cas-cache

pip install django-cas-cache

We'd have some code in a custom Django middleware to update the cache on each page view, looking roughly like this:

middleware.py

from collections import deque
from django.core.cache import cache

class LastSeenMiddleware(object):
    def process_response(request, response):
        # you might want some logic like this to only
        # record successful requests
        if response.status != 200:
            return response

        # in case we don't already have a deque, try to add
        # (add will not overwrite if key already exists)
        added = cache.add('last_seen', deque([request.path], maxlen=20))

        if not added:
            def update(val):
                val.append(request.path)
                return val

            cache.cas('last_seen', update)

        return response

views.py

from django.core.cache import cache
from django.shortcuts import render_to_response

def myview(request):
    last_seen = cache.get('last_seen')
    # whatever
    return render_to_response('mytemplate.html', {'last_seen': last_seen})

settings.py

CACHES = {
    'default': {
        'BACKEND': 'cascache.backends.memcached.MemcachedCASCache',
        'LOCATION': '127.0.0.1:11211',
    }
}

# as a response processor, our middleware should probably come
# first in this list, in order to run *last*
MIDDLEWARE_CLASSES = (
    'myapp.middleware.LastSeenMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
    'django.middleware.security.SecurityMiddleware',
)