1

I use django's built-in caching with the @cache_page decorator. However, I would like the cache to be refreshed periodically automatically, so that a refresh is not triggered by an actual page request from a user resulting in a delay.

An obvious strategy that comes to mind is using a celery task. I have 2 questions please:

  1. If the celery task approach is acceptable, what code do I need to a) trigger the refresh and for b) an unknown number of pages e.g. myapp.com/products/?page=2, myapp.com/products/?page=3 (I cant predict the number of pages)
  2. Is there a better approach?
RunLoop
  • 20,288
  • 21
  • 96
  • 151

2 Answers2

1

I ended up writing a celery task using the requests framework to refresh the cache every hour - it also deals with pagination. Sample code:

@shared_task
def refresh_caches():
   header = {"Content-Type": "application/json; charset=utf-8",
                        "Authorization": settings.USER_TOKEN}

   next_page_url = settings.API_URL +'/products/'
   while len(next_page_url) > 0:
        response = requests.get(next_page_url, headers=header)
        next_page_url = ''
        jsonresponse = response.json()
        if jsonresponse.get('next'):
            next_page_url = jsonresponse['next']
RunLoop
  • 20,288
  • 21
  • 96
  • 151
-1

I think a celery task for that is an overkill. There is a setting that defines the cache expiration:

CACHE_MIDDLEWARE_SECONDS – The number of seconds each page should be cached.

Reference: Django docs

This has been introduced in 1.8 and affects the duration of @cache_page. Do not confuse it with the CACHES: TIMEOUT setting which is for using the cache functions such as cache.set(). More on this here.

Community
  • 1
  • 1
Wtower
  • 18,848
  • 11
  • 103
  • 80
  • I understand the cache timeout. What I want to prevent is a user initiated request refreshing the cache causing a delay - I want to trigger a cache refresh manually. – RunLoop Jul 28 '16 at 06:42
  • @RunLoop I see. My understanding is that a user request shouldn't extend the cache, it is to expire `CACHE_MIDDLEWARE_SECONDS` after is set, but we need to dig in the code to confirm this. This is an issue though when you do not want even that and need to manually expire it. – Wtower Jul 28 '16 at 06:49