remove duplicates from 2d lists regardless of order

Question

I have a 2d list

a = [[1, 2], [1, 3], [2, 1], [2, 3], [3, 1], [3, 2]]

How can I get the result:

result = [[1,2],[1,3],[2,3]]

Where duplicates are removed regardless of their order of the inner lists.

Answer 1

Try using a set to keep track of what lists you have seen:

from collections import Counter

a = [[1, 2], [1, 3], [2, 1], [2, 3], [3, 1], [3, 2], [1, 2, 1]]

seen = set()
result = []
for lst in a:
    current = frozenset(Counter(lst).items())
    if current not in seen:
        result.append(lst)
        seen.add(current)

print(result)

Which outputs:

[[1, 2], [1, 3], [2, 3], [1, 2, 1]]

Note: Since lists are not hash able, you can store frozensets of Counter objects to detect order less duplicates. This removes the need to sort at all.

Answer 2

In [3]: b = []
In [4]: for aa in a:
...:     if not any([set(aa) == set(bb) for bb in b if len(aa) == len(bb)]):
...:         b.append(aa)
In [5]: b
Out[5]: [[1, 2], [1, 3], [2, 3]]

Answer 3

You can try

a = [[1, 2], [1, 3], [2, 1], [2, 3], [3, 1], [3, 2]]
aa = [tuple(sorted(elem)) for elem in a]
set(aa)

output

{(1, 2), (1, 3), (2, 3)}

Answer 4

The 'Set' concept would come in handy here. The list you have (which contains duplicates) can be converted to a Set (which will never contain a duplicate). Find more about Sets here : Set

Example :

l = ['foo', 'foo', 'bar', 'hello']

A set can be created directly:

s = set(l)

now if you check the contents of the list

print(s)
>>> {'foo', 'bar', 'hello'}

Set will work this way with any iterable object! Hope it helps!