Python - how to remove the final comma(,) in json string

Question

Hi I have just started experimenting with python and tornado along with mongodb(I am a newbie). I have written a simple get function to get all the values from my mongodb and return it in JSON format. The problem is when I try to write the output as a JSON string I get a trailing comma(,) after the last record from the collection.

class TypeList(APIHandler):
@gen.coroutine
def get(self):
    cursor = db.vtype.find()
    self.write("{"'"success"'": 1, "'"data"'":[")
    while (yield cursor.fetch_next):
        document = cursor.next_object()
        self.write(format(JSONEncoder().encode(document)))
        self.write(",")
    self.write("]}")

class JSONEncoder(json.JSONEncoder):
def default(self, o):
    if isinstance(o,ObjectId):
        return str(o)
    return json.JSONEncoder.default(self, o)

And my output is like

{"success": 1, "data":[{"_id": "55a5e988545779f35d3ecdf4", "name": "fgkd", "city": "fghj"},{"_id": 12345.0, "name": "adfs", "city": "asd"},]}

Can anyone tell me how can I get rid of that trailing comma(,) after my last record, because of that comma I am getting an error malformed JSON string

I have tried using json dumps

@gen.coroutine
def get(self):
    cursor = db.vtype.find({"brand": "Tata"})
    while (yield cursor.fetch_next):
        document = cursor.next_object()
        self.write(json.dumps(document,default=json_util.default))

got the output as

{"Reg": "11ts", "_id": {"$oid": "55a5e988545779f35d3ecdf4"}, "Name": "Alex"}{"Reg": "12ts", "_id": {"$oid": "55a5eac6545779f35d3ecdf5"}, "Name": "asdf"}

When using dumps[{ "data": document }]

I am getting the output as

[{"data": {"Name": "asdf", "Reg": "asdfs", "_id": {"$oid": "55a5e988545779f35d3ecdf4"}}}]

[{"data": {"Name": "qwer", "Reg": "asdff", "_id": {"$oid": "55a5eac6545779f35d3ecdf5"}}}]

but I want the output like this

{"data": [{"Name": "asdf", "Reg": "asdfs", "_id": {"$oid": "55a5e988545779f35d3ecdf4"}},{"Name": "qwer", "Reg": "asdff", "_id": {"$oid": "55a5eac6545779f35d3ecdf5"}}]}

If I am doing something wrong please tell me I dont know how to do it.

You might want to [look at this](https://docs.python.org/3/library/json.html) — Blakes Seven, Jul 16 '15 at 09:26
Actually while using JSON dumps (json.dumps) I get an error "ObjectId(addsfsfsf)" is not JSON serializable. The ObjectId is from my mongodb — Tony Roczz, Jul 16 '15 at 09:30
I have written the `class JSONEncoder` to overcome that serializer issue. — Tony Roczz, Jul 16 '15 at 09:32
Why re-invent the wheel though; if you figured out the issue with `json.dumps()` you might not have needed to bother with this now. — l'L'l, Jul 16 '15 at 09:34
@I'L'I when using `self.write(json.dumps(document))` getting error `TypeError: ObjectId('55a5e988545779f35d3ecdf4') is not JSON serializable`. Am I using json.dumps the right way pls guide me — Tony Roczz, Jul 16 '15 at 09:47
to overcome the JSON serializable error i [referred to](http://stackoverflow.com/a/16586277/5039470) — Tony Roczz, Jul 16 '15 at 09:58

score 7 · Answer 1 · answered Jul 16 '15 at 09:28

7

There is no reason you should be building up JSON documents via text concatenation.

Python has a perfectly good json module in the standard library which you should be using. Build up your document as a Python list of dicts, then use json.dumps() to convert the whole thing to valid JSON.

answered Jul 16 '15 at 09:28

Daniel Roseman

588,541
66
880
895

1

when using `self.write(json.dumps(document))` getting error `TypeError: ObjectId('55a5e988545779f35d3ecdf4') is not JSON serializable`. Am I using json.dumps the right way pls guide me – Tony Roczz Jul 16 '15 at 09:43
@TonyRoczz The sentiment here is correct. But there is a custom version of "dumps" with your mongodb library. – Blakes Seven Jul 16 '15 at 09:45
to overcome the JSON serializable error i have [referred to](http://stackoverflow.com/a/16586277/5039470) – Tony Roczz Jul 16 '15 at 09:59
Use bson.json_util, from PyMongo, which you've already installed along with Motor. – A. Jesse Jiryu Davis Jul 16 '15 at 14:10

Blakes Seven · Answer 2 · 2015-07-16T11:29:14.533

2

So your problem is with MongoDB ObjectId? Then maybe you should have been using bson.json_util. It's probably already installed as part of your MongoDB driver dependecies ( which all use pymongo ), but if not then install it.

import bson
import bson.json_util
from bson.json_util import dumps
from bson import ObjectId

dumps({ "a": ObjectId() })

'{"a": {"$oid": "55a782261d41c80b0432b811"}}'

Or:

dumps([{ "a": ObjectId(), "b": 1 },{ "a": ObjectId(), "b": 2 }])
'[{"a": {"$oid": "55a79543268e150463d51799"}, "b": 1}, {"a": {"$oid": "55a79543268e150463d5179a"}, "b": 2}]'

And it works just like "dumps" except all the BSON type handling is built it.

Again, no need to re-invent the wheel here and "roll your own", because people already use this.

edited Jul 16 '15 at 11:29

answered Jul 16 '15 at 09:43

Blakes Seven

49,422
14
129
135

can you just post the particular string? I have imported dumps from bson.json_util but it gives me the JSON not serializable error – Tony Roczz Jul 16 '15 at 10:06
@TonyRoczz coded example right there. The output is supported ["extended json"](http://docs.mongodb.org/manual/reference/mongodb-extended-json/) which correctly identifies the "type" of the field. All in the documentation anyway. – Blakes Seven Jul 16 '15 at 10:10
the dumps works fine when I use print instead of self.write – Tony Roczz Jul 16 '15 at 10:27
@TonyRoczz Great. So use it. What the message was here is build your own dicts, use the natural dict responses and let something else handle the serialization. Just like CSV, it's nothing new, so it's all been done before. – Blakes Seven Jul 16 '15 at 10:34
Thanks. the above example works but the issue is the above example does not differentiate the different records of the table the output is like this `{"type": "adfs", "_id": 12345.0, "brand": "asd"}{"type": "adfs", "_id": 12345.0, "brand": "asd"}` there should be a comma inbetween the different records otherwise its not a proper json string. Shouldnt there be a comma after to separate the two records? – Tony Roczz Jul 16 '15 at 11:16
@TonyRoczz You need to wrap it in a "list" otherwise it is not valid JSON. We all know what correctly formatted JSON is here. You might consider from the responses here that it is not "you" who are educating people but the other way around. See the example above. – Blakes Seven Jul 16 '15 at 11:29
Am sorry if i have offended you my intention was not educating you, but I was clarifying my doubt whether what I have understood is right or not, nothing else. since i am new in this field I am finding it difficult take it all in at the same time difficult to understand thats all – Tony Roczz Jul 16 '15 at 13:24

score 0 · Answer 3 · answered Jul 16 '15 at 09:34

0

Your implementation of the JSONEncoder works well. Just use it the way it was intended to be used:

>>> JSONEncoder().encode({'data': [ObjectId(), ObjectId()]})
'{"data": ["<objId>", "<objId>"]}'

The encoder will take care of serializing dicts, objects, lists, tuples, strings (including unicode), ints, longs, floats, booleans and None. Your implementation makes it aware of ObjectIds as well. Perfect!

Just lose the string concatenation and use encode.

answered Jul 16 '15 at 09:34

André Laszlo

15,169
3
63
81

can you kindly guide me on how my code should be formed for this. How do you suggest my `JSONEncoder().encode(document)` statement should be. Thank you in advance – Tony Roczz Jul 16 '15 at 10:11
I think Blake's answer is better, but if you want to keep `JSONEncoder` you can try something like `JSONEncoder().encode({'success': 1, 'data': items})` where `items` is a list of the documents returned by your query. You can probably get them all at once, or use your existing loop to fill an initially empty array. – André Laszlo Jul 16 '15 at 10:36

Python - how to remove the final comma(,) in json string

3 Answers3

Linked