3

I've configured johnny cache with one of my applications that is hosted on apache. It is configured with memcached as the backend which runs on the same machine on the default port.

The caching works fine when multiple web clients go through apache. They all read from the cache and any update is invalidating the cache. But when a python program/script reads from the DB using django (same settings.py that has johnny configuration), it doesn't read from the cache and hence any updates made by that program wont affect the cache. Which leaves me with the web clients reading stale data from the cache.

I haven't found anything in johnny cache's documentation related to this. Any thoughts on this situation?

I'm using johnny cache 0.3.3, django 1.2.5 and python 2.7.

Edit: to answer one of the quetions in the comments, I read from the DB in the script this way-

>>> cmp = MyModelClass.objects.get(id=1)
>>> cmp.cust_field_2
u'aaaa'

I know it doesn't read from the cache because I update the table directly by firing an update sql statement and the updated value is not reflected in my web client as it still reads from the cache. Whereas my script shows the updated value when I re-fetch the object using MyModelClass.objects.get(id=1)

Thanks,

ksrini
  • 1,412
  • 2
  • 19
  • 33
  • Can you post some sample code of how your python script reads from the database? How is this script invoked? Johnny cache relies on a middleware, so if this middleware is somehow being avoided, you won't see values from the cache. – joshcartme Apr 11 '12 at 06:36
  • I simply load an object using - mc = MyModel.objects.get(id=1) and print/view the value on one of its fields – ksrini Apr 11 '12 at 07:28
  • If you restart your webserver does it get the same value as the script? It could be that something else is "caching" things that is unrelated to Johnny. Also is that sample code called via a management command or some other way? How is the script set up? – joshcartme Apr 11 '12 at 08:11
  • No, Restarting the webserver is still giving me the cached result. Which is what I'd expect. My sample script code, I'm just running from the the terminal via the python interactive shell. I have another proper application that also runs on the terminal by a shell script and before it is launched it sets up all env variables needed for django like the settings module and stuff like that. – ksrini Apr 11 '12 at 09:15

2 Answers2

4

It appears that middleware is not called when you run scripts/management comands which is why you are seeing the difference. This makes sense when reading the documentation on middleware because it processes things like request and views, which don't exist in a custom script.

I found a way around this, and there is an issue regarding it in the Johnny Cache bitbucket repo. In your script put the following before you do anything with the database:

from johnny.middleware import QueryCacheMiddleware
qcm = QueryCacheMiddleware()

# put the code for you script here

qcm.unpatch()

You can see more on that here:

https://bitbucket.org/jmoiron/johnny-cache/issue/49/offline-caching

and here:

https://bitbucket.org/jmoiron/johnny-cache/issue/50/johhny-cache-not-active-in-management

joshcartme
  • 2,717
  • 1
  • 22
  • 34
  • I did read about the unpatch thing and did go through the code as well. I interpretted it as - 'do the unpatch if you dont want the caching'. Anyway, I looked at the issues you've mentioned and tried the suggested code. It doesn't seem to help. :( – ksrini Apr 12 '12 at 05:11
  • joshcartme, I'm sorry, it does work! I was just not doing it right. I tried again and it works fine. The documentation and what Jason Moiron says in the issue now makes sense to me. Thank you for your help! – ksrini Apr 12 '12 at 05:47
3

That is the recommended way from the documentation:

from johnny.cache import enable
enable()

Update:

What I observed, as if your tasks.py files have this in the beginning, you can not disable johnny cache using settings.py anymore.

I have reported the issue: https://github.com/jmoiron/johnny-cache/issues/27

hurturk
  • 5,214
  • 24
  • 41