0

I'm attempting to create a python rq worker on heroku for my flask app. The heroku documentation provides the following example code for creating a worker:

https://devcenter.heroku.com/articles/python-rq#create-a-worker

import os

import redis
from rq import Worker, Queue, Connection

listen = ['high', 'default', 'low']

redis_url = os.getenv('REDISTOGO_URL', 'redis://localhost:6379')

conn = redis.from_url(redis_url)

if __name__ == '__main__':
    with Connection(conn):
        worker = Worker(map(Queue, listen))
        worker.work()

Thing is: I am using RedisCloud and replacing REDISTOGO_URL with REDISCLOUD_URL is not working. It's fine loading locally but when I upload to heroku I am seeing in the logs it is still attempting to connect to localhost.

2015-02-25T00:11:53.445167+00:00 app[web.1]: ConnectionError: Error 111 connecting to localhost:6379. Connection refused.

The RedisCloud documentation only lists working with Python, not particularly flask, so the below code did not change my situation at all:

# -*- coding: utf-8 -*-

import os
import urlparse

import redis
from rq import Worker, Queue, Connection

listen = ['high', 'default', 'low']


redis_url = urlparse.urlparse(os.environ.get('REDISCLOUD_URL', 'redis://localhost:6379'))

conn = redis.from_url(redis_url.hostname)

if __name__ == '__main__':
    with Connection(conn):
        worker = Worker(map(Queue, listen))
        worker.work()

i have confirmed that the REDISCLOUD_URL var is present on heroku via:

$heroku config:get REDISCLOUD_URL

in format:

http://rediscloud:password@hostname:port

I can find no documentation on what the redis.from_url() function does or what parameters it accepts.

What am I missing here to get this worker to look to the server var for heroku and not localhost?

chrickso
  • 2,994
  • 5
  • 30
  • 53
  • Are you certain about the 'http' part of your REDISCLOUD_URL? In any case, there's no need to both urlparse and use from_url. Either use the whole url in from_url or, if you other parsing before, open the confection using `conn = redis.Redis(redis_url.hostname, redis_url.port, redis_url.password)` – Itamar Haber Feb 25 '15 at 02:06
  • checking now, i see the REDISCLOUD_URL hostname starts with redis://. what difference does that make? also, i describe above how i attempted both of your suggestions and neither work. – chrickso Feb 25 '15 at 02:10
  • `redis://` at the beginning of a URL denotes the Redis protocol - I'm not sure it makes a difference in your case but 'http://' seems weird. Note that `from_url` accepts a URL-formatted string whereas you're only providing it with the parsed `hostname` in your 2nd example. The connection error you provided indicates that the env var is not returned - perhaps try printing its contents from your code for debugging purposes? – Itamar Haber Feb 25 '15 at 12:51
  • Were you able to find a solution? I'm seeing the same issue right now – GrowinMan Apr 06 '15 at 09:30
  • here's a copy of my worker that i got working: https://gist.github.com/anonymous/766e5ce6c5615e89d539 lools like the fix was needing to create a new instance of redis = Redis() – chrickso Apr 07 '15 at 11:23
  • @chrickso I am facing a similar issue. I tried your gist. But the jobs are queued, never actually run – Ajoy Apr 22 '15 at 10:24
  • 1
    @Ajoy on heroku? make sure you have 2 dyno's running and that your Procfile is starting up your worker.py like this https://gist.github.com/anonymous/31483faa9f1e5cd8204e – chrickso Apr 23 '15 at 01:15

0 Answers0