Django prefetch_related on many to many field always yields empty array

Question

I'm having trouble with Django's prefetch_related.

Here are my models.

class Terrain(models.Model):
    name = models.CharField(max_length=255)

class Route(models.Model):
    name = models.CharField(max_length=255)
    terrains = models.ManyToManyField(Terrain)

And here's what happens when I use prefetch_related.

>>> Route.objects.all()[0].terrains.all()
[<Terrain: Beach>]
>>> Route.objects.all().prefetch_related('terrains')[0].terrains.all()
[]

Any idea what I'm doing wrong here?

Python version 2.7 Django version 1.4 DB Backend: SQLite (I also tried with MySQL, same results)

And here's some query debugging (from IPython console) so you can see what's going on there.

In [11]: Route.objects.all().prefetch_related('terrains')[0].terrains.all()
DEBUG 2012-06-22 20:10:09,569 util (0.000) SELECT "asdf_route"."id", "asdf_route"."name" FROM "asdf_route" LIMIT 1; args=()
DEBUG 2012-06-22 20:10:09,571 util (0.000) SELECT ("asdf_route_terrains"."route_id") AS "_prefetch_related_val", "asdf_terrain"."id", "asdf_terrain"."name" FROM "asdf_terrain" INNER JOIN "asdf_route_terrains" ON ("asdf_terrain"."id" = "asdf_route_terrains"."terrain_id") WHERE "asdf_route_terrains"."route_id" IN (1); args=(1,)
Out[11]: []

Answer 1

Yikes. Figured out the problem. Some time ago there was a bug where prefetch_related wouldn't work if your model's pk field was non-text (like an integer). I manually patched this line:

vals = rel_obj_cache.get(instance_attr_val, [])

near the end of query.py to look like this:

vals = rel_obj_cache.get(unicode(instance_attr_val), [])

That fixed my problem. But since then I guess the problem was solved formally in another way and now my unicode cast was causing that dictionary lookup to return an empty list (basically the reverse of the original bug).

So, just had to undo my manual patch and now it works fine. No wonder I couldn't find anyone else having this problem!