How to pass Non-essential Built-in object as parameter in django-rq enqueue

Question

Update: I found that I could not even pass a class in the enqueue, is this possible to do this?

---

I want to pass a item object (a model in Django) as the parameter in queue.enqueue(grabber.parse_body, item), but the rqworker will raise this Exception (once I tried to pass the item as parameter in spite of whether I use it):

File "/usr/local/lib/python3.4/dist-packages/rq/job.py", line 53, in unpickle raise UnpickleError('Could not unpickle.', pickled_string, e) rq.exceptions.UnpickleError: ('Could not unpickle.', ImproperlyConfigured('Requested setting INSTALLED_APPS, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.',))

Seems that parse_body don't know what the item is?

I know I could just pass the items's id as parameter, and then deal with id in parse_body, but is this possible to use item directly in parse_body?

---

My main code: (enqueue.py and grabber.py are independent script, not in the Django apps.)

enqueue.py

import os
import grabber
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "website.settings.development")

queue = django_rq.get_queue('default')
need_to_parse = Post.objects.all()
    for item in need_to_parse:
        queue.enqueue(grabber.parse_body, item)

grabber.py

def parse_body(item):
    print(item)

Answer 1

The problem is that RQ's default pickler is cPickle which does not know how to serialize django model instances. A simpler approach would be to use model_to_dict and pass a pickable object to your queue.

from django.models import model_to_dict
my_dict = model_to_dict(my_instance,fields=[],exclude=[])

If you are intent on using django model instances in your queue, you can create your own Job class that uses a PickleSerializer first and then set your Queue.job to your new CustomJob(Job). https://docs.djangoproject.com/en/1.7/topics/http/sessions/#bundled-serializers

Answer 2

I had the same error message because I started rqworker with the command

$ rqworker

I fixed this by typing

$ python manage.py rqworker