Comparing three (or more) dictionaries and finding a match if at least two are equal

Question

I'm faced with an issue similar to this one. However, that SO question is strictly focused on three variables. I am looking for a solution that would work for more than three as well.

Here's my code for two variables:

for track_a in collection_a:
    for track_b in collection_b:

        t1 = track_a["tempo"]
        t2 = track_b["tempo"]
        k1 = track_a["key"]
        k2 = track_b["key"]
        m1 = track_a["mode"]
        m2 = track_b["mode"]

        if (t1 == t2) and (k1 == k2) and (m1 == m2):
            collection_c.append((track_a, track_b))

Here's my solution for three variables:

for track_a in collection_a:
    for track_b in collection_b:
        for track_c in collection_c:

            t1 = track_a["tempo"]
            t2 = track_b["tempo"]
            t3 = track_c["tempo"]
            k1 = track_a["key"]
            k2 = track_b["key"]
            k3 = track_c["key"]
            m1 = track_a["mode"]
            m2 = track_b["mode"]
            m3 = track_c["mode"]

            a = (t1 == t2) and (k1 == k2) and (m1 == m2)
            b = (t2 == t3) and (k2 == k3) and (m2 == m3)
            c = (t3 == t1) and (k3 == k1) and (m3 == m1)

            if a: collection_c.append((track_a, track_b))
            if b: collection_c.append((track_b, track_c))
            if c: collection_c.append((track_c, track_a))

Obviously, this solution is not scalable and slow. Considering the fact I'd have to check all of them, I doubt it will ever be fast since we have to iterate over all possible combinations, but could I at least make it scale? (Up to at least 5). Also, if possible, allow more comparison characteristics to be added later.

Answer 1

An efficient approach that solves the issue in linear time is to convert the dicts to frozen sets of key-value tuples (over keys that are used for equality tests) so that they can be hashable and used as dict keys (signatures) themselves, and so that you can simply use a dict of sets to group them:

groups = {}
for track in collections: # collections is a combination of all the collections you have
    groups.setdefault(frozenset((k, track[k]) for k in ('tempo', 'key', 'mode')), set()).add(track['name'])

so that:

[group for group in groups.values() if len(group) >= 3]

will return you a list of sets of names of the 3 tracks whose signatures are identical.

Answer 2

Here is a logically scalable solution that for n dictionaries being compared on m values will take time n*m to evaluate.

Do note, if three match, I will return a group of 3. It is easy enough to then blow that up to all matching pairs. But if you do so, then you could be returning something of size n*n. I have shown you what both look like.

def group_on(variables, *tracks):
    # Build a trie first.
    trie = {}
    for track in tracks:
        this_path = trie
        for variable in variables:
            value = track[variable]
            if value not in this_path:
                this_path[value] = {}
            this_path = this_path[value]
        if 'final' not in this_path:
            this_path['final'] = [track]
        else:
            this_path['final'].append(track)

    def find_groups(this_path, count):
        if 0 == count:
            if 1 < len(this_path['final']):
                yield this_path['final']
        else:
            for next_path in this_path.values():
                for group in find_groups(next_path, count-1):
                    yield group

    for group in find_groups(trie, len(variables)):
        yield group

def group_to_pairs(group):
    for i in range(len(group)-1):
        for j in range(i+1, len(group)):
            yield (group[i], group[j])

print('Efficient version')

for group in group_on(['tempo', 'key', 'mode'],
        {'track': 1, 'tempo': 1, 'key': 'A', 'mode': 'minor'},
        {'track': 2, 'tempo': 1, 'key': 'A', 'mode': 'major'},
        {'track': 3, 'tempo': 1, 'key': 'A', 'mode': 'minor'},
        {'track': 4, 'tempo': 1, 'key': 'A', 'mode': 'major'},
        {'track': 5, 'tempo': 1, 'key': 'A', 'mode': 'minor'},
        ):
    print(group)

print('Versus')

for group in group_on(['tempo', 'key', 'mode'],
        {'track': 1, 'tempo': 1, 'key': 'A', 'mode': 'minor'},
        {'track': 2, 'tempo': 1, 'key': 'A', 'mode': 'major'},
        {'track': 3, 'tempo': 1, 'key': 'A', 'mode': 'minor'},
        {'track': 4, 'tempo': 1, 'key': 'A', 'mode': 'major'},
        {'track': 5, 'tempo': 1, 'key': 'A', 'mode': 'minor'},
        ):
    for pair in group_to_pairs(group):
        print(pair)

Answer 3

Find something useful in itertools, not sure if this is what you want:

from itertools import product, combinations

all_collections = [collection_a, collection_b, collection_c] # d, e, f, ...
for collections in combinations(all_collections, 2):         # Pick 2 (or any number) collections from all collections
    for tracks in product(*collections):                     # Cartesian product of collections or equivalent to for track1 in collection1: for track2 in collection2: ...
        if True:                                             # check if all tracks are matched
            print(*tracks)                                   # or append them to another collection