1

I have the below structure, where content modules, which are subclassed from a common model, are attached to pages via a 'page module' model that references them via a GenericForeignKey:

class SitePage(models.Model):
    title = models.CharField()
    # [etc..]

class PageModule(models.Model):
    page = models.ForeignKey(SitePage, db_index=True, on_delete=models.CASCADE)
    module_type = models.ForeignKey(ContentType, on_delete=models.CASCADE)
    module_id = models.PositiveIntegerField()
    module_object = GenericForeignKey("module_type", "module_id")

class CommonModule(models.Model):
    published_time = models.DateTimeField()

class SingleImage(CommonModule):
    title = models.CharField()
    # [etc..]

class Article(CommonModule):
    title = models.CharField()
    # [etc..]

At the moment, populating pages from this results in a LOT of SQL queries. I want to fetch all the module contents (i.e. all the SingleImage and Article instances) for a given page in the most database-efficient manner.

I can't just do a straight prefetch_related because it "must be restricted to a homogeneous set of results", and I'm fetching multiple content types.

I can get each module type individually:

image_modules = PageModule.objects.filter(page=whatever_page, module_type=ContentType.objects.get_for_model(SingleImage)).prefetch_related('module_object_')
article_modules = PageModule.objects.filter(page=whatever_page, module_type=ContentType.objects.get_for_model(Article)).prefetch_related('module_object')

all_modules = image_modules | article_modules

But I need to sort them:

all_modules.order_by('module_object__published_time')

and I can't because:

"Field 'module_object' does not generate an automatic reverse relation and therefore cannot be used for reverse querying"

... and I don't think I can add the recommended GenericRelation field to all the content models because there's already content in there.

So... can I do this at all? Or am I stuck?

Wintermute
  • 2,973
  • 4
  • 32
  • 52
  • 1
    Check my question and answer [here](https://stackoverflow.com/questions/28127135/is-django-prefetch-related-supposed-to-work-with-genericrelation) I'm not sure if this is gonna solve you the issue with the ordering, but at least you can minimize the number of queries being needed. – Todor Mar 31 '20 at 18:17
  • Hi - thanks, that's useful. I followed it to [this article](https://blog.roseman.org.uk/2010/02/22/django-patterns-part-4-forwards-generic-relations/) which I've based a solution on; it's halved the number of SQL queries. Unfortunately since they're now replaced with `WHERE IN()` those queries aren't terribly more efficient, but it's an improvement at least. – Wintermute Apr 01 '20 at 09:44

1 Answers1

0

Following the advice in the comments above I eventually arrived at this code (from 2012!) that has roughly halved the number of queries: https://gist.github.com/justinfx/3095246

However, as I noted above, it's done that at the expense of creating some fairly inefficient WHERE pk IN() queries, so I've not actually saved much time in total.

Wintermute
  • 2,973
  • 4
  • 32
  • 52