RQ is a simple, lightweight, Python library for creating background jobs, and processing them.
RQ (Redis Queue) is a simple python library for queueing jobs and processing them in the background with workers. It is backed by redis and it is designed to have a low barrier to entry. It should be integrated in your web stack easily.
I have read this stackoverflow Q&A but it did not worked it my case.
In my scenario I push a function (submit_transaction_for_settlement(transaction_id)) to the redis queue using the excellent package django-rq. The job of this function is to submit…
I'm using Python RQ (backed by Redis) to feed tasks to a bunch of worker processes.
I accidentally sent a tuple when adding a job to a queue, so now I have queues like this:
high
medium
low
('low',)
default
I can't seem to figure out how to get rid…
I have followed the tutorial on https://devcenter.heroku.com/articles/python-rq
Let's say I have a function that converts PDF Files to JPG and stores them in the app's tmp/file/ and that it later uploads it to S3.
However, since the request timeout…
I hosted my Django app on Heroku but due to few limitations I moved from Heroku to cloud based server. I followed this tutorial on running background tasks in Python. Everything is running fine except that I have to manually run python worker.py to…
I am implementing python-rq to pass domains in a queue and scrape it using Beautiful Soup. So i am running multiple workers to get the job done. I started 22 workers as of now, and all the 22 workers is registered in the rq dashboard. But after some…
I have a Flask server that is registered to sentry using raven.contrib.flask's Sentry(app, dns=...) object. I also have some python-rq workers for working on async background processes. I found this documentation that seems to state that I can…
I am using python-rq and redis to pass the domain names and get the links from the domain.
try:
for link in [h.get('href') for h in self.soup.find_all('a')]:
--code goes here--
except Exception, ex:
print ex
pass
Whenever i run…
I'm trying to use Python-RQ with Zope by calling an external method (for background tasks) from ZMI after a certain operation. The file called by external method resides in Extensions. It initialises connection to Redis and imports a module that…
I am trying to setup a zmq socket inside a worker-task in python-rq.
If I create the context inside the task function:
def push( user, task_id, data ):
"""Push message to `user` over websocket.
"""
ctx = zmq.Context()
pub =…
I'm trying to run rqworker as an attached-daemon in UWSGI. To do this I add to the config.ini file:
attach-daemon = /path/to/rqworker
Then UWSGI tries to launch rqworker when it should but I get the following error in the logs:
RuntimeError: Click…
My flask app works perfectly locally but when I deploy it to Heroku, it can't connect to the Redis Queue.
THis is the exception I get:
2015-04-06T09:01:45.586197+00:00 app[web.1]: Traceback (most recent call last):
2015-04-06T09:01:45.586189+00:00…
in my flask app, I typically will
from extensions import redis
and then in my functions will do something like
redis.set('redis:test:queue:increment', 1)
and all is fine.
Now that I am attempting to install/understand/use python RQ, I'm finding…
I'm attempting to create a python rq worker on heroku for my flask app. The heroku documentation provides the following example code for creating a worker:
https://devcenter.heroku.com/articles/python-rq#create-a-worker
import os
import redis
from…
Update: I found that I could not even pass a class in the enqueue, is this possible to do this?
I want to pass a item object (a model in Django) as the parameter in queue.enqueue(grabber.parse_body, item), but the rqworker will raise this Exception…
