django: saving a query set to session

Question

I am trying to save query result obtained in one view to session, and retrieve it in another view, so I tried something like below:

def default (request):
    equipment_list = Equipment.objects.all()

    request.session['export_querset'] = equipment_list

However, this gives me

TypeError at /calbase/

<QuerySet [<Equipment: A>, <Equipment: B>, <Equipment: C>]> is not JSON serializable

I am wondering what does this mean exactly and how should I go about it? Or maybe there is alternative way of doing what I want besides using session?

Answer 1

If this is what you are saving:

 equipment_list = Equipment.objects.all()

You shouldn't or wouldn't need to use sessions. Why? Because this is a simple query without any filtering. equipment_list would be common to all the users. This can quite easily be saved in the cache

 from django.core.cache import cache

 equipment_list = cache.get('equipment_list')
 if not equipment_list:
     equipment_list = Equipment.objects.all()
     cache.set('equipment_list',equipment_list)

Note that a queryset can be saved in the cache without it having to be converted to values first.

Update:
One of the other answers mention that a querysets are not json serializable. That's only applicable when you are trying to pass that off as a json response. Isn't applicable when you are trying to cache it because django.core.cache does not use json serialization it uses pickling.

Answer 2

'e4c5' raises a concern which is perfectly valid. From the limited code we can see, it makes no sense to put in the results of that query into the session. Unless ofcourse you have some other plans which we cant quite see here. I'll ignore this and assume you ABSOLUTELY MUST save the query results into the session.

With this assumption, you must understand that the queryset instance which Django is giving you is a python object. You can move this around WITHIN your Django application without any hassles. However, whenever you attempt to send such an entity over the wire into some other data store/application (in your case, saving it into the session, which involves sending this data over to your configured session store), it must be serializable to some format which:

1. your application knows how to serialize objects into
2. the data store at the other end knows how to de-serialize. In this case, the accepted format seems to be JSON. (this is optional, the JSON string can be stored directly)

The problem is, the queryset instance not only contains the rows returned from the table, it also contains a bunch of other attributes and meta attributes which come in handy to you when you use the Django ORM API. When you try to send the queryset instance over the wire to your session store, the system knows no better and tries to serialize all these attributes into JSON. This fails because there are attributes within the queryset that are not serializable into JSON.

As far as a solution is concerned, if you must save data into the session, as some people have suggested, simply performing objects.all().values() and saving it into your session may not always work. A simple case is when your table returns datetime objects. Datetime objects are by default, not JSON serializable.

So what should you do? What you need is some sort of serializer which accepts a queryset, and safely iterates over the returned rows converting each python native datatype into a JSON safe equivalent, and then returning that. In case of datetime.datetime objects, you would need to call obj.isoformat() to transform it into an ISO format datetime string.

Answer 3

You cannot save a QuerySet instance in session, cause well as you said, they're not JSON Serializable. Read This for more information.

To save your queryset, you can use values and values_list methods to get your desired fields, then you cast them to a list and then save the list into session.(most of the time saving only the PKs does the job though).

so basically:

qset = Model.objects.values_list("pk", "field_one", "field_two") # Gives you a ValuesListQuerySet object which's still not serializable.
cache_results = list(qset)
# Now you cache the cache_results variable however you want.
redis.setex("cached:user_id:querytype", 10 * 60, json.dumps(cache_results))

It's also better to change the way you save this special result (values_list) so you can have better lookups, a dictionary might be a good choice.

Answer 4

Saving query sets in django sessions requires them to be serialized and that causes the error. One way of easily moving the query set by saving them in sessions is to make a list of the id of the Equipments model. (Or any other field that serves as the primary key of the model), like:

        equipments = [equipment.id for equipment in Equipment.objects.all()]
        request.session['export_querset'] = equipments

And then whenever you need the Equipments, traverse this list and get the corresponding Equipment.

        equipments = [Equipment.objects.get(id=id) for id in request.session['export_querset']]

Note: This method is inefficient and is not recommended for large query sets, but for small query sets, it can be used without worries.