0

I have something that looks like this:

pages = Page.objects.prefetch_related("sections","sections__tiles").all()
for page in pages:
    for section in page.sections.all():
        for tile in section.tiles.all():
            print tile.author
            # more stuff to build each page

This hits the SQL layer once for the initial query, then once per loop (n+1). However the optimal number of SQL queries is 1 + (count of unique authors).

I implemented a simple hash based cash of "authors", and cut load time dramatically.

cache_author = {}
def author_cache(author_id):
    author = cache_author.get(author_id, None)
    if author:
        return author
    else:
        author = Author.objects.get(id=author_id)
        cache_author[author_id] = author
        return author

pages = Page.objects.prefetch_related("sections","sections__tiles").all()
for page in pages:
    for section in page.sections.all():
        for tile in section.tiles.all():
            print author_cache(tile.author_id)
            # more stuff to build each page

But it feels messy. What cleaner options are out there, for reducing SQL overhead inside a single transaction?

Veedrac
  • 58,273
  • 15
  • 112
  • 169
Bryce
  • 8,313
  • 6
  • 55
  • 73

1 Answers1

0

Again, I can't be positive without seeing the models but I tested this setup in Django 1.7 and it should get everything with only a couple of queries.

pages = Page.objects.prefetch_related("sections","sections__tiles").all().select_related(‌​'sections__tiles__author')
for page in pages:
for section in page.sections.all():
    for tile in section.tiles.all():
        print tile.author
  • Unfortunately that's contorting the code to optimize the SQL. The desired operation here is to build pages: running it the other way would be confusing. – Bryce Sep 25 '14 at 22:27
  • Ahhh, sorry, I missed the comment about building pages in the first example. – daniel tiesling Sep 25 '14 at 22:33
  • Have you tried: pages = Page.objects.prefetch_related("sections","sections__tiles").all().select_related('sections__tiles__author') – daniel tiesling Sep 25 '14 at 22:42