Comparing 2 lists consisting of dictionaries with unique keys in python

Question

I have 2 lists, both of which contain same number of dictionaries. Each dictionary has a unique key. There is a match for each dictionary of the first list in the second list, that is a dictionary with a unique key exists in the other list. But the other elements of such 2 dictionaries may vary. For example:

list_1 = [
            {
                'unique_id': '001',
                'key1': 'AAA',
                'key2': 'BBB',
                'key3': 'EEE'
             },
             {
                'unique_id': '002',
                'key1': 'AAA',
                'key2': 'CCC',
                'key3': 'FFF'
             }
         ]

 list_2 = [
             {
                'unique_id': '001',
                'key1': 'AAA',
                'key2': 'DDD',
                'key3': 'EEE'
             },
             {
                'unique_id': '002',
                'key1': 'AAA',
                'key2': 'CCC',
                'key3': 'FFF'
             }
         ]

I want to compare all elements of 2 matching dictionaries. If any of the elements are not equal, I want to print the none-equal elements.

Would you please help?

score 64 · Accepted Answer · edited Jun 27 '23 at 20:50

64

Assuming that the dicts line up like in your example input, you can use the zip() function to get a list of associated pairs of dicts, then you can use any() to check if there is a difference:

>>> list_1 = [{'unique_id':'001', 'key1':'AAA', 'key2':'BBB', 'key3':'EEE'}, 
              {'unique_id':'002', 'key1':'AAA', 'key2':'CCC', 'key3':'FFF'}]
>>> list_2 = [{'unique_id':'001', 'key1':'AAA', 'key2':'DDD', 'key3':'EEE'},
              {'unique_id':'002', 'key1':'AAA', 'key2':'CCC', 'key3':'FFF'}]
>>> pairs = zip(list_1, list_2)
>>> any(x != y for x, y in pairs)
True

Or to get the differing pairs:

>>> [(x, y) for x, y in pairs if x != y]
[({'key3': 'EEE', 'key2': 'BBB', 'key1': 'AAA', 'unique_id': '001'}, {'key3': 'EEE', 'key2': 'DDD', 'key1': 'AAA', 'unique_id': '001'})]

You can even get the keys which don't match for each pair:

>>> [[k for k in x if x[k] != y[k]] for x, y in pairs if x != y]
[['key2']]

Possibly together with the associated values:

>>> [[(k, x[k], y[k]) for k in x if x[k] != y[k]] for x, y in pairs if x != y]
[[('key2', 'BBB', 'DDD')]]

NOTE: In case your input lists are not sorted yet, you can do that easily as well:

>>> from operator import itemgetter
>>> list_1, list_2 = [sorted(l, key=itemgetter('unique_id')) 
                      for l in (list_1, list_2)]

edited Jun 27 '23 at 20:50

jvperrin

3,368
1
23
33

answered Mar 23 '12 at 19:32

Niklas B.

92,950
18
194
224

@Lattyware: Yes, I'm assuming that the lists are aligned in a way that matching dicts have the same position in both lists. It seemed to me that this was the situation OP faces. – Niklas B. Mar 23 '12 at 19:41
@NiklasB.Definitely, but thought I'd mention it for completeness' sake. – Gareth Latty Mar 23 '12 at 19:43
@Niklas B.: Yes, the lists are sorted. But it might not have been. How would I do it if the lists were not sorted? – alwbtc Mar 23 '12 at 19:44
8

It's worth to mention that ZIP method will works ONLY if we have the same number of dict inside each list. – Doomsday Dec 12 '13 at 14:28
@Doomsday: It's not clear what the output should be if that is not the case, though. – Niklas B. Dec 12 '13 at 16:05
@NiklasB. I would assume that the `dicts` which are missing in `list_1` or list_2` are part of the difference output. – gies0r May 22 '18 at 07:39

gies0r · Answer 2 · 2021-03-06T22:10:05.400

18

The fastest and most comprehensive way would be, to use two sets of tuples:

set_list1 = set(tuple(sorted(d.items())) for d in sorted(list1))
set_list2 = set(tuple(sorted(d.items())) for d in sorted(list2))

(if your list is already sorted, simply remove the list sort to save performance)

Find overlapping using intersection:

set_overlapping = set_list1.intersection(set_list2)

Find difference using symmetric_difference

set_difference = set_list1.symmetric_difference(set_list2)

Convert tuple back to dict

 for tuple_element in set_difference:
     list_dicts_difference.append(dict((x, y) for x, y in tuple_element))

edited Mar 06 '21 at 22:10

answered May 23 '18 at 10:40

gies0r

4,723
4
39
50

@gies0r How should I change the last part of the code for overleaping? I just want to get matching keys not values! – eabanoz Mar 01 '19 at 04:41
This assumes list1 and list2 are sorted already, you need to call sort on `set_list1` and `set_list2` – Mark Mar 03 '21 at 10:17
@Mark: Was working on sorted list from the code - Thanks - Edited. – gies0r Mar 06 '21 at 22:10

score 3 · Answer 3 · answered Mar 23 '12 at 19:36

3

The following compares the dictionaries and prints the non-equal items:

for d1, d2 in zip(list_1, list_2):
    for key, value in d1.items():
        if value != d2[key]:
            print key, value, d2[key]

Output: key2 BBB DDD. By using zip we can iterate over two dictionaries at a time. We then iterate over the items of the first dictionary and compare the value with the corresponding value in the second dictionary. If these are not equal, then we print the key and both values.

answered Mar 23 '12 at 19:36

Simeon Visser

118,920
18
185
180

How does it know that it should compare dictionaries with the same "unique_id" key? – alwbtc Mar 23 '12 at 19:46
1

Based on the example I assumed the list with dictionaries was ordered. If that's not the case then you need to order it first by `unique_id`. – Simeon Visser Mar 23 '12 at 19:48

score 1 · Answer 4 · answered Apr 28 '15 at 21:12

I have a version that actually does not depends on a particular key, so the elements are equal (zero) or they are not (non-zer):

list_1 = [{'unique_id':'001', 'key1':'AAA', 'key2':'BBB', 'key3':'EEE'}, {'unique_id':'002', 'key1':'AAA', 'key2':'CCC', 'key3':'FFF'}]
list_2 = [{'unique_id':'001', 'key1':'AAA', 'key2':'DDD', 'key3':'EEE'}, {'unique_id':'002', 'key1':'AAA', 'key2':'CCC', 'key3':'FFF'}]
list_3 = [{'Name': 'Abid', 'Age': 27},{'Name': 'Mahnaz', 'Age': 27}]
list_4 = [{'Name': 'Abid', 'Age': 27},{'Name': 'Mahnaz', 'Age': 27}]

print cmp(list_1,list_1)
print cmp(list_1,list_3)
print cmp(list_1,list_2)
print cmp(list_2,list_1)
print cmp(list_3,list_4)

gives:

Return Value :  0
Return Value :  1
Return Value : -1
Return Value :  1
Return Value :  0

what does ```cmp()``` do? can you provide me the implementation of ```cmp()``` — Shubham Srivastava, Dec 14 '17 at 18:40
@ShubhamSrivastava: you can read it from the official info "cmp(x, y) Compare the two objects x and y and return an integer according to the outcome. The return value is negative if x < y, zero if x == y and strictly positive if x > y". https://docs.python.org/2/library/functions.html#cmp — alemol, Dec 15 '17 at 19:25

score 0 · Answer 5 · answered Apr 25 '17 at 02:38

Let list1 = []
list2 = []

To fetch all the key values we can do like this:
key_values = list1[0]
key = key_values.keys() //key is a list and contains all key values

below is a piece of code which compares all the key pair values:

for val in list1:
    first_key = key[0]
    for val2 in list2:
        if val2[first_key] == val[first_key]:
            for val3 in key:
                if val2[val3] != val[val3]:
                    Unmatched.append(val)

print unmatched

Above contains matches dictionary and prints for which all key, pair values didn't match.

score 0 · Answer 6 · answered Jan 06 '18 at 05:09

def new_change(old_list, new_list):
    change_list = []
    for x in new_list:
        for y in old_list:
            if x['unique_id'] != y['unique_id']:
                change_list.append(x)
    return change_list

pass old and new list in side of this method

Mohd Shoaib · Answer 7 · 2023-02-23T10:26:08.070

compare 2 list of dict on basis of key value and get the different dict items from another::-------

db_items = [
{
    "id": "db...1111",
    "name": "amit",
    "type": "cricket",
    "value": "kumar"
},
{
    "id": "db...222",
    "name": "rishabh",
    "type": "traveller",
    "value": "gupta"
},
{
    "id": "items.....2222",
    "name": "raj",
    "type": "foodie",
    "value": "ankur"
}
]
items = [
{
    "id": "items.....1111",
    "name": "abhishek",
    "type": "cricket",
    "value": "sharma"
},
{
    "id": "items.....2222",
    "name": "hardik",
    "type": "traveller",
    "value": "sharma"
},
{
    "id": "db...333",
    "name": "shanchay",
    "type": "music",
    "value": "kumar"
}
]
for item in items.copy():
for i in db_items:
    if item.get("type") == i.get("type"):
        items.remove(item)

print(items)

output:---
[{'id': 'db...333', 'name': 'shanchay', 'type': 'music', 'value': 
'kumar'}]

Comparing 2 lists consisting of dictionaries with unique keys in python

7 Answers7

Linked

Related