How to match substring in a list of nested dictionary in python?

Question

I am stuck with the below issue where I am trying to search for specific value in a nested dictionary inside of a nested list. It doesn't have to completely match but more like contains. Below is the example data

data=[[{'A': 'test1'}, {'A': 'test2'}, {'BB': {'A': '111testabc'}}], 
 [{'A': 'test1'}, {'A': 'test3'}, {'BB': {'A': '999000abc'}}], 
 [{'A': 'test1'}, {'A': 'test4'}, {'BB': {'A': '99111testabc123'}}], 
 [{'A': 'test1'}, {'A': 'test5'}, {'BB': {'A': '123456abc'}}]]

I want to extract all the nested list including dictionary. The serach string is 111testabc. So my expected output would be

[[{'A': 'test1'}, {'A': 'test2'}, {'BB': {'A': '111testabc'}}],
 [{'A': 'test1'}, {'A': 'test4'}, {'BB': {'A': '99111testabc123'}}]]

I am trying to solve this with:

key, value = 'A', '111testabc'
for lp in data:
    dictList.append([myDict for myDict in lp if myDict.get(key) in value])

Can anyone please let me know how to resolve this? I am using python3.9

score 1 · Answer 1 · answered Dec 13 '22 at 15:23

Assuming you do not know how deep the dictionary goes, you would need to write a recursive function. Something along the lines of

import re

def search_string(dictionary, search_value):
    results=[]
    for key in dictionary.keys():
        if type(dictionary[key])==dict:
            search_string(dictionary[key]
        else:
            search_result=re.search(search_value, dictionary[key])
            if len(search_results) > 0:
                results.append(search_result)
            else: 
                continue

    return search_results

Then you can go over your list of dicts and you should get a list of lists

mysearch=[search_string(my_dict, <search_value>) for my_dict in my_list]

If you have a list of lists of dicts then you run the loop twice. If you do not know how deep the list goes you would need to write another recursive function for your lists like so:

def recursive_list(my_list):
   recursive_list_results=[]
   for item in my_list:
     if type(item)==list:
        recursive_list(item)
     else:
        results=search_string(item, search_value)
        recursive_list_results.append(results)

   return recursive_list_results

P.S. check my whitespace it might not be consistent and I did not test the code. you might need to debug it a bit.

score 1 · Answer 2 · answered Dec 13 '22 at 15:44

As @celalps answer solves your task, I will not talk about that.

However, I would like to add something about your approach, since there are 2 issues:

a)

for lp in data:
    dictList.append([myDict for myDict in lp if myDict.get(key) in value])

The list comprehension iterates over all elements in lp, but none of these in your example has the key A. That is why you need to go one layer deeper or why celalp suggests a recursive approach.

b)

if myDict.get(key) in value

will not yield what you want. You at least need to turn it around:

if value in myDict.get(key)

though this results in an error if the key does not exist.

score 1 · Answer 3 · answered Dec 13 '22 at 17:34

Well, you could define a recursive helper function that returns true under three conditions:

The value is a string that contains match
The value is a list that contains match
The value is a dictionary that contains match (as a value)

data = [
    [{"A": "test1"}, {"A": "test2"}, {"BB": {"A": "111testabc"}}],
    [{"A": "test1"}, {"A": "test3"}, {"BB": {"A": "999000abc"}}],
    [{"A": "test1"}, {"A": "test4"}, {"BB": {"A": "99111testabc123"}}],
    [{"A": "test1"}, {"A": "test5"}, {"BB": {"A": "123456abc"}}],
]

def find_value(value, match):
    value_type = type(value)
    return (
        value_type == str and match in value
        or value_type == list and any(find_value(sub_value, match) for sub_value in value)
        or value_type == dict and any(find_value(sub_value, match) for sub_value in value.values())
    )

match = "111testabc"
result = [lis for lis in data if find_value(lis, match)]
for lis in result:
    print(lis)

Output:

[{'A': 'test1'}, {'A': 'test2'}, {'BB': {'A': '111testabc'}}]
[{'A': 'test1'}, {'A': 'test4'}, {'BB': {'A': '99111testabc123'}}]

Suggestions?

How to match substring in a list of nested dictionary in python?

3 Answers3