0

I have a bit of code that looks something like this.

page_objects = Page.objects.exclude(lorem).filter(blah).annotate(key=bar).order_by(foo)
paginator = Paginator(page_objects, 50)

On my UI, I notice that with certain values of bar, I am missing rows in the output. I ran an experiment to compare the output of paginator against that of the query.

page_results = []
for page_number in paginator.page_range:
    result = paginator.page(page_number)
    page_results += result.object_list
paginator_ids = sorted([x.id for x in page_results])
page_object_ids = sorted(x.id for x in list(page_objects))
print(paginator_ids == page_object_ids) => FALSE
print(len(paginator_ids) == len(page_object_ids)) => TRUE

As you can see, the list of objects in the query and the list of all objects in the paginator did not match. I visually inspected the lists to confirm that this is correct (the ids in the two lists are clearly different).

I am on Django==1.11.3

What is going on and why is my paginator not behaving like the docs?

Interestingly, if I change the order_by to order_by(foo, 'id'), the paginator behaves as expected.

wonton
  • 7,568
  • 9
  • 56
  • 93

1 Answers1

0

the paginator simply slices the output of the query. That combined with the lazy queries returns the data in an order that is not necessary the same as the entire query. In short there is nothing wrong with the paginator or the query set, you are simply comparing two lists that are ordered in a different way(as you noticed by adding the id).

tstoev
  • 1,415
  • 11
  • 12
  • If it's an ordering problem, running this bit of code 100 times would eventually give me `True` but that's not the case. The problem here is that the sum of the slices deterministically lacks some rows. – wonton Nov 07 '19 at 17:48
  • can you pull the missing records only? is there a pattern – tstoev Nov 07 '19 at 19:46