How to convert nested dictionary with numpy array to json and back?

Question

I am new to the python world, below is my nested dictionary with values as NumPy array and I want to convert it to JSON, and convert it back to the nested dictionary with NumPy array from JSON. Actually, I am trying to convert it using json.dumps() but it is giving me an error that says: the object of type ndarray is not JSON serializable

{'protein': 
   {'chicken': array([112.5 ,  90.  ,  67.5 ,  45.  ,  22.5 ,  11.25,   0.  ]),
    'banana': array([200., 150., 100.,  50.,  25.,   0.]),
    'carrots': array([2.35 , 1.88 , 1.41 , 0.94 , 0.47 , 0.235, 0.   ])
   },
  'carbohydrate': 
     {'chicken': array([0., 0., 0., 0., 0., 0., 0.]),
      'banana': array([200., 150., 100.,  50.,  25.,   0.]),
      'carrots': array([17. , 13.6, 10.2,  6.8,  3.4,  1.7,  0. ])}, 
   'fat': 
      {'chicken': array([13. , 10.4,  7.8,  5.2,  2.6,  1.3,  0. ]),
       'banana': array([1.56 , 1.17 , 0.78 , 0.39 , 0.195, 0.   ]), 
       'carrots': array([0.6 , 0.48, 0.36, 0.24, 0.12, 0.06, 0.  ])
      }
}

If you use the encoder from this answer, it should work: https://stackoverflow.com/questions/50916422/python-typeerror-object-of-type-int64-is-not-json-serializable/57915246#57915246 — shimeji42, Dec 18 '20 at 09:08

score 5 · Accepted Answer · answered Dec 18 '20 at 09:07

Going only one way - from any Python structure to JSON - is easy; pass a default= function to json.dumps.

from numpy import array
import json

data = {
    "protein": {
        "chicken": array([112.5, 90.0, 67.5, 45.0, 22.5, 11.25, 0.0]),
        "banana": array([200.0, 150.0, 100.0, 50.0, 25.0, 0.0]),
        "carrots": array([2.35, 1.88, 1.41, 0.94, 0.47, 0.235, 0.0]),
    },
    "carbohydrate": {
        "chicken": array([0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]),
        "banana": array([200.0, 150.0, 100.0, 50.0, 25.0, 0.0]),
        "carrots": array([17.0, 13.6, 10.2, 6.8, 3.4, 1.7, 0.0]),
    },
    "fat": {
        "chicken": array([13.0, 10.4, 7.8, 5.2, 2.6, 1.3, 0.0]),
        "banana": array([1.56, 1.17, 0.78, 0.39, 0.195, 0.0]),
        "carrots": array([0.6, 0.48, 0.36, 0.24, 0.12, 0.06, 0.0]),
    },
}

def convert(x):
    if hasattr(x, "tolist"):  # numpy arrays have this
        return x.tolist()
    raise TypeError(x)

print(json.dumps(data, default=convert))

However, if you need to also be able to roundtrip back to the exact structure with nested Numpy arrays, you will some more magic involving the object_hook callback for json.loads:

def convert(x):
    if hasattr(x, "tolist"):  # numpy arrays have this
        return {"$array": x.tolist()}  # Make a tagged object
    raise TypeError(x)


def deconvert(x):
    if len(x) == 1:  # Might be a tagged object...
        key, value = next(iter(x.items()))  # Grab the tag and value
        if key == "$array":  # If the tag is correct,
            return array(value)  # cast back to array
    return x


json_data = json.dumps(data, default=convert)
data2 = json.loads(json_data, object_hook=deconvert)

This way you have a convention that JSON objects of the shape {"$array": [1, 2, 3]} should be interpreted as Numpy arrays.

Shouldnt it allways raise error? Except when this is not a numpy array? — adir abargil, Dec 18 '20 at 09:25
I mean what does the convert function does while encoding? Isnt it iterating any inner objects? — adir abargil, Dec 18 '20 at 10:15
The `default` function will be called for every object that the JSON library doesn't know how to convert, so if you return a list of unconvertable objects, the JSON library takes care of calling the function again. — AKX, Dec 18 '20 at 10:59

How to convert nested dictionary with numpy array to json and back?

1 Answers1