Python: Convert Cyclical dictionary to JSON

Question

I'm trying to convert a nested cyclical dictionary to JSON. I am getting an overflow error:

In [8]: xx = json.dumps(d)
---------------------------------------------------------------------------
OverflowError                             Traceback (most recent call last)
<ipython-input-8-95e57b3e2ca3> in <module>()
----> 1 xx = json.dumps(d)

OverflowError: Maximum recursion level reached

Not sure why this is happening, but my guess is that it has something to do with my dictionary, and how it's structured.

How did you get the *nested cyclical dictionary*? – Moses Koledoye Jul 20 '16 at 20:16 — Moses Koledoye, Jul 20 '16 at 20:16
Hmm, I get a `ValueError` with a cyclical dictionary. – Alex Jul 20 '16 at 20:20 — Alex, Jul 20 '16 at 20:20

score 0 · Answer 1 · answered Jul 20 '16 at 20:20

Python's json decoder does have a feature to check for cyclical objects - check_circular - which defaults to True. Its behavior is exactly to raise the overflowerro you are seeing.

(In Python 3.5 I get a ValueError with check_circular enabled and a RecursionError with it disabled)

Now, setting it to False certainly won't fix things, since a JSON representation of a cyclical data structure would still be infinite.

The only way is to create a custom JSON encoder and DECODER, and devise a schema for your own decoder to be able to restore any cyclical reference.

score 0 · Answer 2 · answered Jul 20 '16 at 20:54

How to make a cyclical dictionary, shortest example I can think of:

>>> foo = { 'bar': None }
>>> foo['bar'] = foo
>>> foo
{'bar': {...}}
>>> foo['bar']
{'bar': {...}}
>>> foo['bar']['bar']
{'bar': {...}}

So the question is what your question is. Despite the fact Python (at least 2.7, anyway) allows the cyclical reference, what do you want to do with it? Do you really want the JSON data to be able to support a cycle? It seems impractical to create your own encoder and decoder - then it's not really JSON, it's not data you can generically pass to others as JSON, a decoder written to standards wouldn't be able to decode it properly.

It seems to make far more sense to find and eliminate the cycles, or somehow self refer without actually self referring in the dictionary, perhaps create some sort of reference class, perhaps, that can find the item in the dictionary you are looking for using a list of keys and indexes (for lists) instead of referring to the object directly?

And then run it through a standard encoder.

Python: Convert Cyclical dictionary to JSON

2 Answers2