2

Goal

Querying for all products, slicing them, returning subset of those products with an added key:value , in other words, enriched.

Code that works but I can't use

I can't use this code because I use a paginator, the paginator accesses the count of the QuerySet. If I pass the sliced QuerySet then that count is just for that sliced part, not the overall QuerySet, hence why I can't use it. 

products_qs = final_qs[paginator.get_offset(request):
                       paginator.get_offset(request) + paginator.get_limit(request)]
for product in products_qs:
    product.raw['super_cool_new_key'] = ms_response.get('results').get(product.id)

This works great, when I print the data I can see that super_cool_new_key enrichment in every product. Awesome. Problem? Well, I have had to slice it and now the count method is no longer true. Of course, I can do something like:

products_qs.count = final_qs.count

and move on with my life, but it feels... hacky, or maybe not?

Code I would like for it to work, but doesn't

for i in range(paginator.get_offset(request),
               paginator.get_offset(request) + paginator.get_limit(request)):
    product = final_qs[i]
    product.raw['super_cool_new_key'] = ms_response.get('results').get(product.id)

When I see the output of the data, the super_cool_new_key is not there. I can't wrap my head around as to why?

Maybe I am having a thick day and I don't understand accessing by reference, so I remove the middlemonkey.

final_qs = final_qs.all()
for i in range(paginator.get_offset(request),
               paginator.get_offset(request) + paginator.get_limit(request)):
    final_qs[i].raw['super_cool_new_key'] = ms_response.get('results').get(final_qs[i].id, '')

Suspicions

It's obvious it's something about the code difference that is the culprit of why one way works and the other way doesn't. My dollar is on the following:

  • The slice
  • The iteration

Looking into Django Docs for QuerySet :

Iteration. A QuerySet is iterable, and it executes its database query the first time you iterate over it.

Then about slicing:

Slicing. As explained in Limiting QuerySets, a QuerySet can be sliced, using Python’s array-slicing syntax. Slicing an unevaluated QuerySet usually returns another unevaluated QuerySet, but Django will execute the database query if you use the “step” parameter of slice syntax, and will return a list

I can't be the slicing then, because I don't do a slice with a "step" parameter. Since it returns an unevaluated QuerySet the code I want to work, should in theory work. (Isn't that always the case?ha ha)

Ok so that clears up the fact that when, in the first option of coding, I did an iteration of for x in x_container the QuerySet was executed. Could that be the answer? So I modified the code:

Spoiler Alert: still does not work

final_qs = final_qs.all()
for i in range(paginator.get_offset(request),
               paginator.get_offset(request) + paginator.get_limit(request)):
    product = final_qs[i]
    product.raw['super_cool_new_key'] = ms_response.get('results').get(product.id)

Emmm... help?

A suggested answer that, spoiler alert, did not work

from django.db.models import When, Case, Value, CharField
when = [ When(id=k, then=Value(v)) for k,v in ms_response.get('results').items()]
p = final_qs[paginator.get_offset(request)
             :paginator.get_offset(request) + paginator.get_limit(request)]
p = p.annotate(super_cool_new_key=Case(
            *when,
            default=Value(''),
            output_field=CharField()
        )
    )

I also tried it without slicing but with .all().annotate() . Still didn't work. It doesn't work, not due to an Exception happening, but because when I see the output, that super_cool_new_key is not there, meaning it didn't enrich the objects, which is the whole point.

Chayemor
  • 3,577
  • 4
  • 31
  • 54
  • would you consider accessing the ms_response in the template? So anytime you need the results you would just access the ms_response based on the product.id in the template instead of worrying about merging the two. That way your QuerySet stays a queryset and you dont have to worry about pagination – grrrrrr Feb 13 '19 at 14:23
  • Not possible in this current project, there is no template, a single endpoint that needs all this info nicely packed into JSON. – Chayemor Feb 14 '19 at 15:26

1 Answers1

0

It sounds like what you are looking for is similar to the answer here which utilizes When and Case. For your use it will be something along the following:

from django.db.models import When, Case, Value, CharField

ms_response = {5458: 'abc', 9900: 'def'}
whens = [
    When(id=k, then=Value(v)) for k, v in ms_response.items()
]
qs = YourModelName.objects.all().annotate(
    super_cool_key=Case(
        *whens,
        default=Value('xyz'),
        output_field=CharField()
    )
)

when you then call qs.get(id=5458).super_cool_key it will then return 'abc'

grrrrrr
  • 1,395
  • 12
  • 29