How to reload a whole queryset from db in django?

Question

I'm running periodic tasks with celery. One such task gets objects from db with by filter:

pages = Page.objects.filter(active=True)

initially(before starting celery itself) there are 10 such objects in db with active=True. The task executes accordingly when started, on all these 10 objects.

from django.db import transaction
from celery.task import task
from celery import group
from .models import Page

#@transaction.atomic
@task()
def check_pages():
    #@transaction.commit()
    pages = Page.objects.filter(active=True) #not updated later
    g = group(page_check.s(page.id) for page in pages)
    g()


#settings.py
app.conf.beat_schedule = {
    'run-every-1-second': {
        'task': 'tasks.check_pages',
        'schedule': 1.0,
    },
}

Now, if I add another Page object with active=True, this is not recognized or filtered in the task(which is already running periodically).

I know obj.reload_from_db(). But that is applicable only to reload an object and not a <QuerySet>.

I also tried using transaction.commit() with @transaction.atomic. But that stops tasks' execution since transaction doesn't work over multiple workers and tasks.

What am I doing wrong here?

This code wouldn't have the behaviour you describe. Each invocation of `check_pages` would get a fresh queryset from the database. The problem must be somewhere else. — Daniel Roseman, Feb 13 '19 at 09:31

Jrog · Answer 1 · 2021-09-22T03:07:30.267

8

Use all() method. That make copy of queryset without cached result. Check this doc

So you can

@task()
def check_pages():
    #@transaction.commit()
    pages = Page.objects.filter(active=True).all() # now it will update later
    g = group(page_check.s(page.id) for page in pages)
    g()

edited Sep 22 '21 at 03:07

answered Feb 13 '19 at 11:35

Jrog

1,469
10
29

How to reload a whole queryset from db in django?

1 Answers1