How to access redis connection in background task

Question

I'm trying to extend the flask-base project https://github.com/hack4impact/flask-base/tree/master/app which comes with a user model only. I'm trying to add the ability to run a background task on redis using rq. I've found https://devcenter.heroku.com/articles/python-rq which is helpful.

this app has support for redis queues with a background redis queue being implemented by running :

@manager.command
def run_worker():
    """Initializes a slim rq task queue."""
    listen = ['default']
    conn = Redis(
        host=app.config['RQ_DEFAULT_HOST'],
        port=app.config['RQ_DEFAULT_PORT'],
        db=0,
        password=app.config['RQ_DEFAULT_PASSWORD'])

    with Connection(conn):
        worker = Worker(map(Queue, listen))
        worker.work()

using:

$ python manage.py run_worker

In my views I have:

@main.route('/selected')
def background_selected():
    from rq import Queue
    from manage import run_worker.conn
    q = Queue(connection=conn)
    return q.enqueue(selected)

The problem is I don't know how to import the connection created in run_worker() into my view. I've tried variations of :

from manage import run_worker.conn

but I'm getting:

SyntaxError: invalid syntax.

How can I get access to the conn variable in the background task?

score 1 · Accepted Answer · answered Feb 12 '18 at 19:11

from the documentation, python-rq Configuration

Can you try by making the below changes:

manager.py

import redis

"""Initializes a slim rq task queue."""
listen = ['default']
conn = redis.Redis(host=app.config['RQ_DEFAULT_HOST'], 
                   port=app.config['RQ_DEFAULT_PORT'],
                   db=0,     
                   password=app.config['RQ_DEFAULT_PASSWORD'])

@manager.command
def run_worker():
    with Connection(conn):
        worker = Worker(map(Queue, listen))
        worker.work()

and from view:

from rq import Queue
from manage import conn

q = Queue(connection=conn)

Thanks for your help. I didn't realize it before but there is a flask addon called flask-rq (https://pythonhosted.org/Flask-RQ/) which provides a @job decorator to allow background tasks to be run. — user1592380, Feb 13 '18 at 15:57

score 1 · Answer 2 · answered Mar 03 '18 at 20:08

1

I contacted the developer who provided the following:

answered Mar 03 '18 at 20:08

user1592380

34,265
92
284
515

How to access redis connection in background task

2 Answers2