python package for traversing data structures

Question

I spent hours trying to figure out how to subclass and modify json.JSONEncoder to override how it encodes enum.Enum instances before learning that it literally wasn't possible, and that the only solution is to traverse my data structure and convert the enums myself in a pre-processing step.

Then I tried finding an existing solution to the problem of traversing arbitrary Python data structures, which I assumed was a very common and well-solved problem (either in Standard Library or PyPI), but I can't seem to find any.

What exactly are you trying to achieve? What use case do you have for traversal? What would a JSON translation of an enum look like? It's unclear what problems you're having and what your question is - were you just posting an opinion on Python? If you're asking for recommendations, know that that's not what StackOverflow is for (but also that it's not clear what you're even looking for exactly). — Grismar, Oct 24 '22 at 23:19
I believe my question is pointed enough to justify being here - I'm looking for a package that implements recursive traversal of Python data structures and calls a function for each object as it traverses it so I can convert enums. The reason I need it is because the default behavior of json.JSONEncoder is to convert an enum item to item.value and I want item.name instead. — odigity, Oct 24 '22 at 23:24
It seems to me that no routine could traverse any arbitrary data type, so I'm assuming you want it to traverse only standard data types like `dict`, `list`, `tuple`, etc. (or types conforming to some type like iterable, supporting `__get_item__`, etc.)? Such a routine would be fairly straightforward to write in a few lines of Python, which is probably why there isn't such a package - especially since it depends on what you want to do during the traversal, what the optimal procedure would be. What would you want to convert enums to? It sounds like you ultimately want to serialise to json? — Grismar, Oct 24 '22 at 23:29
It looks like a very simple task. You need to insert your code to understand what didn't work for you. Or at least how your data looks like. Minimal running example — Nikolay Zakirov, Oct 24 '22 at 23:31
I want to convert the enum to it's `.name` property instead of it's `.value` property, which is the default. (The types may vary and aren't relevant.) — odigity, Oct 24 '22 at 23:32
Note that the question was closed because it looks like you're asking for recommendations - but as the answer below shows, you may have just missed the obvious solution with a JSONEncoder. If the answer as given doesn't solve your problem and you had some specific problem with this type of solution, you can add information to that effect to the question and get it reopened. (just a friendly tip: saying you "spent hours" doesn't generally help in getting answers - more so if the solution seems somewhat trivial to those reading the question, then it just seems like you didn't really try at all) — Grismar, Oct 25 '22 at 02:29

Grismar · Answer 1 · 2022-10-25T02:31:23.727

You didn't provide an example of what you're trying to achieve, but from the description, it appears you were after something like this:

from json import JSONEncoder, dumps
from enum import Enum


class Color(Enum):
    RED = 1
    GREEN = 2
    BLUE = 3


d = {'c': Color.RED, 'xs': [1, 3.14, Color.BLUE]}


class EnumNameJSONEncoder(JSONEncoder):
    def default(self, obj):
        if isinstance(obj, Enum):
            return obj.name
        return super().default(obj)


r = EnumNameJSONEncoder().encode(d)
print(dumps(r))

Output:

"{\"c\": \"RED\", \"xs\": [1, 3.14, \"BLUE\"]}"

If this doesn't solve your problem, you should provide a more specific example of what you're trying to achieve, what you tried and where you got stuck.

Edit: your comment suggested that the default() doesn't get called for enums - but that's not correct. For example, if you change the class like this:

class EnumNameJSONEncoder(JSONEncoder):
    def default(self, obj):
        print(f'getting called for {type(obj)}')
        if isinstance(obj, Enum):
            return obj.name
        return super().default(obj)

The output becomes:

getting called for <enum 'Color'>
getting called for <enum 'Color'>
"{\"c\": \"RED\", \"xs\": [1, 3.14, \"BLUE\"]}"

This won't work because default will never get called for an enum because it's treated as a recognized type (str or int), but I appreciate you trying. — odigity, Oct 25 '22 at 02:28
It's working just fine in this example - can you provide an example where it doesn't work? — Grismar, Oct 25 '22 at 02:31

python package for traversing data structures

1 Answers1