Json dumping bytes fails in Python 3

Question

I am sending binary data in a post request as part of a request. I have a dictionary that looks like this:

data = {"foo": "bar", "bar": b'foo'}

When I try to json.dumps this dictionary, I get the following exception:

TypeError: b'foo' is not JSON serializable

This worked fine in Python 2.7. What do I have to do to json encode this data?

Possible duplicate of [TypeError: b'1' is not JSON serializable](http://stackoverflow.com/questions/24369666/typeerror-b1-is-not-json-serializable) — Hayley Guillou, Nov 19 '15 at 03:56

Michael Recachinas · Answer 1 · 2015-11-19T04:09:18.240

In Python 3, they removed byte support in json. (Source: https://bugs.python.org/issue10976).

A possible workaround is:

import json

data = {"foo": "bar", "bar": b"foo"}

# decode the `byte` into a unicode `str`
data["bar"] = data["bar"].decode("utf8")

# `data` now contains
#
#   {'bar': 'foo', 'foo': 'bar'}
#
# `json_encoded_data` contains
#
#   '{"bar": "foo", "foo": "bar"}'
#
json_encoded_data = json.dumps(data)

# `json_decoded_data` contains
#
#   {'bar': 'foo', 'foo': 'bar'}
#
json_decoded_data = json.loads(data)

# `data` now contains
#
#   {'bar': b'foo', 'foo': 'bar'}
#
data["bar"] = data["bar"].encode("utf8")

If you don't have a constraint of using json, you might consider using bson (Binary JSON):

import bson

data = {"foo": "bar", "bar": b"foo"}

# `bson_encoded_data` contains 
#
#   b'\x1f\x00\x00\x00\x05bar\x00\x03\x00\x00\x00\x00foo\x02foo\x00\x04\x00\x00\x00bar\x00\x00'
#
bson_encoded_data = bson.BSON.encode(data)

# `bson_decoded_data` contains
#
#   {'bar': b'foo', 'foo': 'bar'}
#
bson_decoded_data = bson.BSON.decode(data)

score 4 · Answer 2 · answered Jun 24 '19 at 10:15

With Json module you cannot dump bytes. A suitable alternative is to use Simple Json Module.

To Install Simple json:

pip3 install simplejson

Code:

import simplejson as json
data = {"foo": "bar", "bar": b"foo"}
json.dumps(data)

Hopefully you won't get the error now!

score 0 · Answer 3 · answered Jun 15 '22 at 23:06

The cls argument of json.dump lets you specify more encoding rules. For handling bytes, you can say:

import json

class ByteEncoder(json.JSONEncoder):
    def default(self, x):
        return x.decode('utf-8') if isinstance(x, bytes) else super().default(x)

print(json.dumps({'a': 'aa', 'b': b'bb'}, indent=1, cls=ByteEncoder))

Json dumping bytes fails in Python 3

3 Answers3