is it possible to avoid n+1 django query here? If so, how?

Question

I have these models

class Thing1(MyModel):
    thing2 = models.OneToOneField(
        Thing2, on_delete=models.PROTECT, related_name="super_pack"
    )
    some_id = models.IntegerField()


class Thing2(MyModel):
    name = models.CharField(max_length=50, primary_key=True)

class Thing3(MyModel):
    name = models.CharField(max_length=255)
    thing2 = models.ForeignKey(
        Thing2,
        related_name="thing3s",
    )

class Thing4(MyModel):
    root_thing3 = models.OneToOneField(
        Thing3, on_delete=models.PROTECT, related_name="my_related_name"
    )
    member_item_thing3s = models.ManyToManyField(
        Thing3,
        through="unimportant",
        related_name="important_related_name",
    )
    is_default = models.BooleanField(default=False)

I'm working with a Django serializer. Already defined I have a queryset with prefetching.

As it stands, the following is performant:

all_thing1s: QuerySet[Thing1]
for thing1 in all_thing1s:
    first_thing3 = Thing1.thing2.thing3s.all()[0]

(from now on assume we are in the loop body)

which is not ideal, if there is always 1 thing3 the schema shouldn't allow for many, but I can't change the schema at this time.)

At this point I believe the ability to pre-fetch further has been broken because we have returned a Thing3 (first_thing3) and no longer have a query set to work with At the start of this I thought Django magically used previous prefetches, I didn't realize that the filters etc. that you wanted to make use of the prefetches had to be chained to the prefetches.

But now I want to do:

thing4 = first_thing3.important_related_name.all()[0]
return thing4.root_thing3

(note: first_thing3 and root_thing3 share the same type but are not the same. And yes, again, the same silly assumption that there is a single thing4 related to first_thing3)

But as per the bold text, I believe prefetching on the view's queryset is impossible.

I thought that perhaps you could be like

first_thing3 = Thing1.thing2.thing3s.all()[0]
query_set = Thing1.thing2.thing3s.filter(id=first_thing.id)
query_set.important_related_name[0]
...

But no because the addition of the filter would have to be account for in the original prefetch which is impossible, and the subsequent important_related_name only makes sense if it is on a single item anyway.

So this was about 8 hours of discovery for me. I think I have determined that it is impossible to prefetch using Django here and that n+1 is unavoidable.

Can a queryset by an aggregation?

I'm looking to confirm that avoiding n+1 is impossible (using built in Django functionality).

is it possible to avoid n+1 django query here? If so, how?

0 Answers0