Filtering and grouping in Python Model Objects after querying the database

Question

I have the following model:

class Item(models.Model):
    user = models.ForeignKey(settings.AUTH_USER_MODEL, blank=True, null=True, related_name='item',on_delete=models.SET_NULL)
    email = models.BooleanField(blank=True, null=True)
    .....

I need to send an email to user, where Item email attribute is False.

I have the following query:

 items = Item.objects.filter(email=False)

I want to filter/group item by User. I want to do it python because I want to send emails with delay and not to overwhelm the database at once with many queries.

For each use I want to create a context dictionary that I can sent to the Django Email.

A pseudo code/logic:

  user_context { 
      'subject' : 'some subject'}

 for item in items:
   if user is the same:
    user_context['content] = append item

    if finished same_user items:
      send_email(user_context)
      context = {} # reset context

But here a `User` can have *zero*, *one* or *multiple* `Item`s. What should happen in case *no* or *multiple* `Item`s exist for a `User`? — Willem Van Onsem, Sep 01 '18 at 12:06
if multiple group items and send them all at the same time(this is what I'm trying to do). The item cannot be created without an User. — user3541631, Sep 01 '18 at 12:21

Willem Van Onsem · Accepted Answer · 2018-09-01T12:42:25.320

You can use itertools.groupby to group an iterable into an iterable of 2-tuples. In order to process this effectively, it is however important to do some prefetching of (related) User objects, like:

from operator import itemgetter
from itertools import groupby

qs = Item.objects.filter(email=False).order_by('user_id').prefetch_related('user')

result = [
    (k, list(vs))
    for k, v in groupby(qs, key=attrgetter('user'))
]

Now result is a list of 2-tuples: the first element of every tuple contains the related User object, the second item of a tuple the list of relevant Item objects (with email=False). Users with no such Items will not be in the list.

So in case you want to send an email, you can loop over the list of 2-tuples like:

for user, items in result:
    user_context { 
        'subject' : 'some subject'
    }
    user_context['content] = items
    send_email(user_context)

If you are however only intersted in the Users where there exist an Item with email=False, then you only need to query:

User.objects.filter(item__email=False)

I update my question with a pseudo code/logic to be more clear what I'm trying to do — user3541631, Sep 01 '18 at 12:29

Filtering and grouping in Python Model Objects after querying the database

1 Answers1