I have something that looks like this:
pages = Page.objects.prefetch_related("sections","sections__tiles").all()
for page in pages:
for section in page.sections.all():
for tile in section.tiles.all():
print tile.author
# more stuff to build each page
This hits the SQL layer once for the initial query, then once per loop (n+1). However the optimal number of SQL queries is 1 + (count of unique authors).
I implemented a simple hash based cash of "authors", and cut load time dramatically.
cache_author = {}
def author_cache(author_id):
author = cache_author.get(author_id, None)
if author:
return author
else:
author = Author.objects.get(id=author_id)
cache_author[author_id] = author
return author
pages = Page.objects.prefetch_related("sections","sections__tiles").all()
for page in pages:
for section in page.sections.all():
for tile in section.tiles.all():
print author_cache(tile.author_id)
# more stuff to build each page
But it feels messy. What cleaner options are out there, for reducing SQL overhead inside a single transaction?