Python (3.4) Recursion function not being called on dict/tree flattening

Question

I'm trying to flatten a nested dictionary while also swapping out its keys according to another nested dict. I assume that none of the branches will have identical keys. E.g.:

In:

values_dict_test = {"abs": 3, "cd": 23, "sdf": "abc", "gxr":
{"rea": 21, "bdf": 95}}
mapping_dict_test = {"abs": "one", "cd": "two", "sdf": "three", "gxr":
{"rea": "four", "bdf": "five"}}

Expected Out:

{"one": 3, "two": 23, "three": "abc", "four": 21, "five": 95}

I'm using the iteritems hack to try to make this code compatible with Python 2.7, but am testing on 3.4. I've added a bunch of print statements to trace execution; it appears the recursive call never actually happens.

try:
    dict.iteritems
except AttributeError:
    # Python 3
    def itervalues(d):
        return iter(d.values())
    def iteritems(d):
        return iter(d.items())
else:
    # Python 2
    def itervalues(d):
        return d.itervalues()
    def iteritems(d):
        return d.iteritems()

def flatten_dict(mapping, values_dict):
    print("Function called with {} and {}".format(mapping, values_dict))
    for (key, value) in iteritems(mapping):
        print("K: {} V: {} Mapping: {}".format(key, value, mapping))
        if isinstance(value, dict):
            # Note that we have to flatten the values_dict as we flatten
            # mapping dict, hence the [key]
            print("Going to recurse")
            print("Passing {} and {}".format(value, values_dict[key]))
            flatten_dict(value, values_dict[key])
        else:
            print("Going to yield {}, {}".format(value, values_dict[key]))
            yield (value, values_dict[key])

values_dict_test = {"abs": 3, "cd": 23, "sdf": "abc", "gxr":
{"rea": 21, "bdf": 95}}
mapping_dict_test = {"abs": "one", "cd": "two", "sdf": "three", "gxr":
{"rea": "four", "bdf": "five"}}

for (x,y) in flatten_dict(mapping_dict_test, values_dict_test):
    print(x, y)

OUTPUT:

Function called with {'cd': 'two', 'sdf': 'three', 'abs': 'one', 'gxr': {'rea': 'four', 'bdf': 'five'}} and {'cd': 23, 'sdf': 'abc', 'abs': 3, 'gxr': {'rea': 21, 'bdf': 95}}
K: cd V: two Mapping: {'cd': 'two', 'sdf': 'three', 'abs': 'one', 'gxr': {'rea': 'four', 'bdf': 'five'}}
Going to yield two, 23
two 23
K: sdf V: three Mapping: {'cd': 'two', 'sdf': 'three', 'abs': 'one', 'gxr': {'rea': 'four', 'bdf': 'five'}}
Going to yield three, abc
three abc
K: abs V: one Mapping: {'cd': 'two', 'sdf': 'three', 'abs': 'one', 'gxr': {'rea': 'four', 'bdf': 'five'}}
Going to yield one, 3
one 3
K: gxr V: {'rea': 'four', 'bdf': 'five'} Mapping: {'cd': 'two', 'sdf': 'three', 'abs': 'one', 'gxr': {'rea': 'four', 'bdf': 'five'}}
Going to recurse
Passing {'rea': 'four', 'bdf': 'five'} and {'rea': 21, 'bdf': 95}

(Normally, I'd use values = {key: value for (key, value) in values_dict_gen} where values_dict_gen is the generator returned by that function.)

EDIT: In favor of reopening, (1) the duplicate linked to uses a mutable default argument, which is known to behave counter-intuitively in these situations (2) it's older, and the answer given below shows a Python 3.3 solution I didn't see on the older question.

score 4 · Accepted Answer · answered Apr 22 '15 at 20:07

flatten_dict(value, values_dict[key])

This call does the same thing inside a generator as it would outside; it creates a generator object. It does not automatically run the generator and yield its items. If you want to use a generator recursively, you have to iterate over it yourself:

for item in flatten_dict(value, values_dict[key]):
    yield item

or in Python 3:

yield from flatten_dict(value, values_dict[key])

Python (3.4) Recursion function not being called on dict/tree flattening

1 Answers1