TypeError can't pickle function objects (Django caching)

Question

Why does the first example throw TypeError (can't pickle function objects) and the second one doesn't, I suppose it's got to do with QuerySet evaluation (Django 1.4)?

def get_or_set_foo_cache():
    if not cache.get('foo'):
        foo = Foo.objects.annotate(bar_count=Count('bar')).filter(bar_count__gte=1)
        print type(foo) # prints <class 'django.db.models.query.QuerySet'>
        cache.set('foo', foo, 60 * 15)
        return foo
    return cache.get('foo')

Example 2

def get_or_set_foo_cache():
    if not cache.get('foo'):
        foo = Foo.objects.all()
        print type(foo) # prints <class 'django.db.models.query.QuerySet'>
        cache.set('foo', foo, 60 * 15)
        return foo
    return cache.get('foo')

If I set foo with list comprehension it works:

foo = [obj for obj in Foo.objects.annotate(bar_count=Count('bar')).filter(bar_count__gte=1)]

What are you pickling exactly? Are you sure `cache` doesn't have a `foo` when you call example 2? — Pierre GM, Sep 15 '12 at 14:04
I even get this error if I try to set the cache directly e.g. cache.set('foo', Foo.objects.annotate(bar_count=Count('bar')).filter(bar_count__gte=1)) — Hedde van der Heide, Sep 17 '12 at 14:59

score 6 · Accepted Answer · answered Sep 18 '12 at 10:36

Queryset is not a list of result objects. It is a lazily evaluated objects that runs its query when you first try to read its content. But when you print it from the console its output is same as list of objects. So most people think them as simple lists.

At the second example you are converting your queryset to a list. Thats why it works. You could also do

foo = list(Foo.objects.annotate(bar_count=Count('bar')).filter(bar_count__gte=1))

And it would work too.

If you won't need them as Foo objects later on. I suggest to use values or values_list operator. Which will give faster results and smaller memory footprint. ( http://www.yilmazhuseyin.com/blog/dev/django-orm-performance-tips-part-2/ )

That's what I thought from reading the docs, I tried adding .all() to the query which didn't seem to trigger evaluation — Hedde van der Heide, Sep 18 '12 at 10:44

TypeError can't pickle function objects (Django caching)

1 Answers1