How to serialize ObjectId with jsonpickle in flask pymongo

Question

I'm using Flask with MongoDB as a database and pymongo as a driver. I have a trouble serializing object into JSON with json.dumps(), json.loads(), json_util.dumps() and many more as it always says TypeError: Object of type 'EachRoom' is not JSON serializable. I found a library called jsonpickle to serialize object into JSON. It's been done well as expected except for the ObjectId fields. It shows as a bytes type when i check the datatype. This is my code:

class EachRoom:
    def __init__(self, id, rooms):
        self.id =  id
        self.rooms = rooms

class Room(object):
    @classmethod
    def return_all_by_user(cls):
        cursor_user = mongo.db.rooms.distinct('user_id')
        users = []

        for row in cursor_user:
            print(str(row))
            cursor_room = mongo.db.rooms.find({'user_id': ObjectId(row)})

            each = EachRoom(ObjectId(row), cursor_room)
            users.append(each)
            print(cursor_room)
        print(users)
        user = jsonpickle.encode(users, unpicklable=False)
        res_user = jsonpickle.decode(user)
        print(res_user)

        if cursor_room:
            return Response(
                json_util.dumps({'message': 'success', 'user': res_user}, json_options=json_util.RELAXED_JSON_OPTIONS),
                mimetype='application/json'
            )
        else:
            return {'message': 'Document rooms is empty'}, 404

And here's the response:

{
    "message": "success",
    "user": [
        {
            "id": {
                "$binary": {
                    "base64": "Xl/WQvzhBgBTGeiE",
                    "subType": "00"
                }
            },
            "rooms": [
                {
                    "_id": {
                        "$binary": {
                            "base64": "XngkNuL9Y6AIX04v",
                            "subType": "00"
                        }
                    },
                    "created_at": "2020-03-23 09:51:34.789000",
                    "name": "Ruang Makan",
                    "updated_at": "2020-03-23 09:51:34.789000",
                    "user_id": {
                        "$binary": {
                            "base64": "Xl/WQvzhBgBTGeiE",
                            "subType": "00"
                        }
                    }
                },
                {
                    "_id": {
                        "$binary": {
                            "base64": "XngkROL9Y6AIX04w",
                            "subType": "00"
                        }
                    },
                    "created_at": "2020-03-23 09:51:48.834000",
                    "name": "Kamar Mandi",
                    "updated_at": "2020-03-23 09:51:48.834000",
                    "user_id": {
                        "$binary": {
                            "base64": "Xl/WQvzhBgBTGeiE",
                            "subType": "00"
                        }
                    }
                },
                {
                    "_id": {
                        "$binary": {
                            "base64": "XngkTOL9Y6AIX04x",
                            "subType": "00"
                        }
                    },
                    "created_at": "2020-03-23 09:51:56.220000",
                    "name": "Kamar Tidur Utama",
                    "updated_at": "2020-03-23 09:51:56.220000",
                    "user_id": {
                        "$binary": {
                            "base64": "Xl/WQvzhBgBTGeiE",
                            "subType": "00"
                        }
                    }
                }
            ]
        },
        {
            "id": {
                "$binary": {
                    "base64": "XnYgKnLe9u63KLbG",
                    "subType": "00"
                }
            },
            "rooms": [
                {
                    "_id": {
                        "$binary": {
                            "base64": "Xnq+2G5/pNeS7ynf",
                            "subType": "00"
                        }
                    },
                    "created_at": "2020-03-25 09:15:52.819000",
                    "name": "Kamar Tidur Eren",
                    "updated_at": "2020-03-25 09:15:52.819000",
                    "user_id": {
                        "$binary": {
                            "base64": "XnYgKnLe9u63KLbG",
                            "subType": "00"
                        }
                    }
                },
                {
                    "_id": {
                        "$binary": {
                            "base64": "Xnq+6W5/pNeS7yng",
                            "subType": "00"
                        }
                    },
                    "created_at": "2020-03-25 09:16:09.683000",
                    "name": "Kamar Tidur Utama",
                    "updated_at": "2020-03-25 09:16:09.683000",
                    "user_id": {
                        "$binary": {
                            "base64": "XnYgKnLe9u63KLbG",
                            "subType": "00"
                        }
                    }
                },
                {
                    "_id": {
                        "$binary": {
                            "base64": "Xnq+8G5/pNeS7ynh",
                            "subType": "00"
                        }
                    },
                    "created_at": "2020-03-25 09:16:16.262000",
                    "name": "Kamar Mandi",
                    "updated_at": "2020-03-25 09:16:16.262000",
                    "user_id": {
                        "$binary": {
                            "base64": "XnYgKnLe9u63KLbG",
                            "subType": "00"
                        }
                    }
                }
            ]
        }
    ]
}

The only thing that goes wrong is the id, user_id, and _id as the ObjectId type. Need advice please..

Check this [JSON serializing Mongodb](https://stackoverflow.com/questions/19674311/json-serializing-mongodb). Maybe you are interested in [MongoEngine](http://docs.mongoengine.org/tutorial.html) which fits better your code. MongoEngine has method `to_json` which serializes correctly BSON objects (ObjectId, DbRef, etc...) — Valijon, Mar 25 '20 at 15:51
@Valijon thanks, but according to [this answer](https://stackoverflow.com/a/41519867/7162428) that he doesn't recommend mongoengine as it is no longer maintained, too buggy, and has a lot of unexpected behavior. Should i? — Vicky Sultan, Mar 25 '20 at 15:59
It's a personal opinion, no proof provided. It's one of officially recognised [MongoDB driver](https://github.com/MongoEngine/mongoengine) and mantained actively by community — Valijon, Mar 25 '20 at 17:06

score 1 · Accepted Answer · answered Mar 25 '20 at 17:06

Here's three tips in one:

To load a class object into MongoDB, use the class's built-in __dict__ property.
To turn your BSON into JSON use dumps from the bson.json_util module
Finally avoid using id as a variable; it's a python function.

Here's a small sample to try:

from pymongo import MongoClient
from bson import ObjectId
from bson.json_util import dumps

db = MongoClient()['mydatabase']

class EachRoom:
    def __init__(self, room_id, rooms):
        self.id = room_id
        self.rooms = rooms

room1 = EachRoom(ObjectId("123456789012345678901234"), '1')
db.rooms.insert_one(room1.__dict__)
print(dumps(db.rooms.find_one()))

Gives:

{"_id": {"$oid": "5e7b8d7ebf681e2e5ecb6059"}, "id": {"$oid": "123456789012345678901234"}, "rooms": "1"}

Wow, i was helped by your `__dict__` hint, although the rest are irrelevant for my code. Thank you for the tips! — Vicky Sultan, Mar 26 '20 at 16:07

How to serialize ObjectId with jsonpickle in flask pymongo

1 Answers1