generator to recursively find all occurrences of a specific object type in a arbitrary nesting of dicts and lists

Question

I want to find all instances of str in any nested data structure of dicts and lists. Not all terminal items will be str.

A data example:

data = {'iso_seq_bams': [['5/X/tmp2oWhu5.tmp', 'y/H/tmp6Po0_X.tmp']], 
        'annotation': None, 
        'bams': {'BAM': {'ERR579132Aligned.sortedByCoord.out.bam': ['Y/o/tmpntzREn.tmp', 'z/c/tmp6DmQhS.tmp']}, 
                 'INTRONBAM': {}}}

And the result expected would thus be ['5/X/tmp2oWhu5.tmp', 'y/H/tmp6Po0_X.tmp', 'Y/o/tmpntzREn.tmp', 'z/c/tmp6DmQhS.tmp']

I have attempted to implement this in a recursive fashion, but it doesn't seem to work. The result is an empty list, currently.

def descend_object(obj):
    if isinstance(obj, dict):
        for item in obj.values():
            descend_object(item)
    elif isinstance(obj, list):
        for item in obj:
            descend_object(item)
    elif isinstance(obj, str):
        yield obj

related to http://stackoverflow.com/questions/10569636/how-to-get-all-keysvalues-in-nested-dict-of-list-of-dicts-and-dicts?rq=1 — ᴀʀᴍᴀɴ, Feb 06 '17 at 19:16
Yielding from the recursive calls leads to a nested group of generators, without actually flattening the result — Ian Fiddes, Feb 06 '17 at 19:17

juanpa.arrivillaga · Accepted Answer · 2017-02-06T19:19:19.133

If on Python 3, use yield from

In [3]: def descend_object(obj):
   ...:     if isinstance(obj, dict):
   ...:         for item in obj.values():
   ...:             yield from descend_object(item)
   ...:     elif isinstance(obj, list):
   ...:         for item in obj:
   ...:             yield from descend_object(item)
   ...:     elif isinstance(obj, str):
   ...:         yield obj
   ...:

In [4]: list(descend_object(data))
Out[4]:
['Y/o/tmpntzREn.tmp',
 'z/c/tmp6DmQhS.tmp',
 '5/X/tmp2oWhu5.tmp',
 'y/H/tmp6Po0_X.tmp']

If on Python 2, you have to manually iterate over the recursive call:

In [6]: def descend_object(obj):
   ...:     if isinstance(obj, dict):
   ...:         for item in obj.values():
   ...:             for d in descend_object(item):
   ...:                 yield d
   ...:     elif isinstance(obj, list):
   ...:         for item in obj:
   ...:             for d in descend_object(item):
   ...:                 yield d
   ...:     elif isinstance(obj, str):
   ...:         yield obj
   ...:

In [7]: list(descend_object(data))
Out[7]:
['Y/o/tmpntzREn.tmp',
 'z/c/tmp6DmQhS.tmp',
 '5/X/tmp2oWhu5.tmp',
 'y/H/tmp6Po0_X.tmp']

I am developing for python 2.7, unfortunately. Is there a 2.7 equivalent to this? — Ian Fiddes, Feb 06 '17 at 19:18

generator to recursively find all occurrences of a specific object type in a arbitrary nesting of dicts and lists

1 Answers1