Django performance with M2M

Question

I have a model that looks like this:

class ParentObject(models.Model):
  ...
  services = models.ManyToManyField(Service)

class ChildObject(models.Model):
  parent = models.ForeignKey(ParentObject)
  services = models.ManyToManyField(Service)

class Service(models.Model):
  name = ...
  description = ...

So, in summary, an object has a list of Services attached and a child objects has to have other Services attached.

In size, each Parent has a list of 50-60 Services and each Child (5 per Parent) has a list of 30-40 Services. In django-admin (using grappelli btw) I have set Child to inline.

The problem is that the page in admin loads very hard (3-5 seconds) because Django Admin executes about 1200 queries (to get the Service each time - sometimes multiple times) to show me the information to edit.

Do you know any tips/tricks to optimize this?

Thank you in advance.

score 2 · Answer 1 · answered Jul 03 '11 at 15:29

2

I haven't been playing with grappelli but in standard django-admin I would consider usage of: ModelAdmin.raw_id_fields. The limitation is that you don't select services using name but using pk.

By default, Django's admin uses a select-box interface () for fields that are ForeignKey. Sometimes you don't want to incur the overhead of having to select all the related instances to display in the drop-down.

raw_id_fields is a list of fields you would like to change into an Input widget for either a ForeignKey or ManyToManyField:

class ArticleAdmin(admin.ModelAdmin):
    raw_id_fields = ("newspaper",)

Harder approach would be to override admin manager for Service and add request level cache. Remember to make this manager available in case of elated objects access.

answered Jul 03 '11 at 15:29

Munhitsu

1,014
1
9
16

this approach offered a significant performance boost for a situation very similar to rsavu's. We combine this approach with the use of automatic model instance caching and invalidation via [django-cache-machine](http://jbalogh.me/2010/02/09/cache-machine/) – Archie1986 Jul 06 '11 at 18:49
@Archie1986 do you know if django-cache-machine is fully compatible with django 1.3? I might need to use it soon. – Munhitsu Jul 06 '11 at 19:28
We use it with Django 1.3 and we have not encountered any technical issues as of yet. One should also note that there are subtleties to be aware of which make django-cache-machine not so plug-and-play (e.g. re: cache invalidation) so read the documentation thoroughly. – Archie1986 Jul 12 '11 at 16:05

score 1 · Answer 2 · answered Jul 03 '11 at 22:09

You need to override the admin queryset method and add a select_related in it, put this in your admin.py file

class ServiceAdmin(admin.ModelAdmin):
    ...

    def queryset(self, request):
        qs = super(ServiceAdmin, self).queryset(request)
        return qs.select_related()

admin.site.register(Service, ServiceAdmin)

This alone should decrease your queries a little bit, but the main problem is that the Django select_related doesn't follow ManyToMany automatically. You need to do some extra workaround for that. I would use the prefill_entry_list from the FeinCMS application to do that.

Django performance with M2M

2 Answers2