Storing a calculation in a flask route cache

Question

I have a long task that goes off into a python-rq worker queue.

@cache.cached(timeout=2592000)
@app.route('/as/<keyword>', methods=['GET'])
@db_session
def auto_suggest(keyword):
    job = q.enqueue(find_keyword, keyword)
    while not job:
        time.sleep(1)
    return jsonify(word=job)

I'm using flask-cache to try and save the result as its a calculation that only needs to be run once a week at most, once a month is sufficient as well.

The problem I am having is when this runs, it caches the failed response, and not the actual solution.

Any other way to do this? Or suggestion in how I should approach this?

Why put the job into a queue when you then just wait for the job? And the job object is not the result, which is why `jsonify(word=job)` causes a failed response each time. — Martijn Pieters, Dec 23 '15 at 18:32

Martijn Pieters · Accepted Answer · 2015-12-24T08:46:54.597

2

You are trying to JSON-encode the Job proxy object, not the result of the job. You do want to check for the job result here:

job = q.enqueue(find_keyword, keyword)
while not job.result:
    time.sleep(1)
return jsonify(word=job.result)

This should be cached just fine.

You could instead memoize the result of the find_keyword function, using the Cache.memoize() decorator:

@cache.memoize(30 * 24 * 60 * 60)
def find_keyword(keyword):
    # ....

This will cache the results of the find_keyword() function for a month, based entirely on the keyword argument (results for different keywords are cached independently).

One more note: when applying extra decorators to a view function, make sure you put the @app.route() decorator at the top (to be applied last). Any decorator applied 'later' is ignored as it is not their output that is being registered as the view:

@app.route('/as/<keyword>', methods=['GET'])
@cache.cached(timeout=2592000)
@db_session
def auto_suggest(keyword):
    job = q.enqueue(find_keyword, keyword)
    while not job.result:
        time.sleep(1)
    return jsonify(word=job.result)

edited Dec 24 '15 at 08:46

answered Dec 23 '15 at 18:35

Martijn Pieters

1,048,767
296
4,058
3,343

Quick question regarding the ```cache.memoize```, would I try to cache that within the python rq worker since that is where the find_keyword function is? And will that be saved if the worker restarts for what ever reason? – nadermx Dec 23 '15 at 19:47
@nadermx: It depends on where your cache is located. If you cache in memcached or redis then your cache will survive a restart. If you use the simple cache, then that's only kept in memory of the process, so restarting the worker would see the cache cleared. – Martijn Pieters Dec 23 '15 at 22:41
1

By the way, the `route` decorator should be the outermost. – lord63. j Dec 24 '15 at 02:09
@lord63.j: yes, indeed; otherwise the cache decorator is ignored. – Martijn Pieters Dec 24 '15 at 08:45

Storing a calculation in a flask route cache

1 Answers1