Get list of unique multi-sets

Question

How can I uniquify the following list in Python:

all_the_ways = [(5,), (2, 2, 1), (2, 1, 2), (2, 1, 1, 1), (1, 2, 2),\
                (1, 2, 1, 1), (1, 1, 2, 1), (1, 1, 1, 2), (1, 1, 1, 1, 1)]

Desired output is:

[(5,), (2, 2, 1), (2, 1, 1, 1), (1, 1, 1, 1, 1)]

i.e. I need to get rid of tuples that have the same set of numbers but in different order.

I tried

set(all_the_ways)

but it only transpose elements.

And when I do

list(map(set, all_the_ways))

the things getting only worse:

[{5}, {1, 2}, {1, 2}, {1, 2}, {1, 2}, {1, 2}, {1, 2}, {1, 2}, {1}]

In other words I need to convert inner tuple to a collection that allows multiple equal items (set is not suitable) and for which permutations of elements don't change the collection itself (kinda like C++'s multiset)

What should be the output when `all_the_ways = [(2, 1, 2), (2, 2, 1)]`? — thefourtheye, Apr 29 '14 at 07:45

YS-L · Accepted Answer · 2014-04-29T07:07:10.970

5

How about this:

list(set(tuple(sorted(s)) for s in all_the_ways))

Output:

[(1, 2, 2), (5,), (1, 1, 1, 1, 1), (1, 1, 1, 2)]

It will mangle the order of each tuple though. I'm assuming that doesn't matter, as tuples containing the same set of numbers are considered the same for your case. What this implies is that in the end, the output list might contain tuples that are not among the original input, for example (credit to @thefourtheye):

all_the_ways = [(2, 1, 2), (2, 2, 1)]
# Output: [(1, 2, 2)]

This may or may not be a problem, and if it is, you can use the more robust solutions which are already mentioned in the other excellent answers.

edited Apr 29 '14 at 07:07

answered Apr 29 '14 at 05:55

YS-L

14,358
3
47
58

1

If the combination `(1, 2, 2)` doesn't exist in `all_the_ways`, this might be a problem. But not sure if it is okay with the OP. Already +1ed – thefourtheye Apr 29 '14 at 06:28
That's very true, as I have mentioned in the answer. I have decided not to address the ordering issue to provide a simpler perspective, just in case it is not a constraint in this problem. +1s to all the order-preserving solutions! :) – YS-L Apr 29 '14 at 06:51
1

Actually, its not about the order. When `all_the_ways = [(2, 1, 2), (2, 2, 1)]`, output will be `[(1, 2, 2)]`, which is not there in `all_the_ways`. This might be a problem, I guess. – thefourtheye Apr 29 '14 at 06:54
Thanks for the example. I actually see it as another "internal ordering" issue there. – YS-L Apr 29 '14 at 07:02
Looks like [OP is fine with this output](http://stackoverflow.com/questions/23356512/get-list-of-unique-multi-sets?noredirect=1#comment35778162_23356512) :-) – thefourtheye Apr 29 '14 at 09:10

Raymond Hettinger · Answer 2 · 2014-04-29T06:26:11.720

3

Use collections.Counter() to identify the unique multisets:

>>> from collections import Counter

>>> all_the_ways = [(5,), (2, 2, 1), (2, 1, 2), (2, 1, 1, 1), (1, 2, 2),\
                (1, 2, 1, 1), (1, 1, 2, 1), (1, 1, 1, 2), (1, 1, 1, 1, 1)]
>>> result = []
>>> seen = set()
>>> for tup in all_the_ways:
        key = tuple(sorted(Counter(tup).items())) # unique signature
        if key not in seen:
            result.append(tup)
        seen.add(key)

>>> result
[(5,), (2, 2, 1), (2, 1, 1, 1), (1, 1, 1, 1, 1)]

edited Apr 29 '14 at 06:26

answered Apr 29 '14 at 05:55

Raymond Hettinger

216,523
63
388
485

I was thinking about this only, but couldn't proceed further as they are not hashable... :( – thefourtheye Apr 29 '14 at 05:58
@thefourtheye Once the counting is made, a sort of the items makes canonical ordering, and tuplizing makes it hashable :-) – Raymond Hettinger Apr 29 '14 at 06:01
Why not simply `Counter(tuple(sorted(i)) for i in all_the_ways).keys()` ? – Jayanth Koushik Apr 29 '14 at 06:01
@RaymondHettinger Actually I used `frozenset` and solved it, but sacrificed the order :( Please check my answer. – thefourtheye Apr 29 '14 at 06:02
1

Anyway +1 to yours, as it maintains the order :) – thefourtheye Apr 29 '14 at 06:04

thefourtheye · Answer 3 · 2014-04-29T06:32:14.840

If the Order doesn't matter you can use this

from collections import Counter
>>> {frozenset(Counter(tup).items()):tup for tup in data}.values()
# [(1, 2, 2), (1, 1, 1, 2), (1, 1, 1, 1, 1), (5,)]

If you want to maintain the Order,

from collections import Counter, OrderedDict
OrderedDict([frozenset(Counter(tup).items()),tup] for tup in data).values()
# [(5,), (1, 2, 2), (1, 1, 1, 2), (1, 1, 1, 1, 1)]

In both the solutions we rely on frozenset, because the set objects are not hashable as they are mutable. In the first case, we construct a dictionary with the frequency of the numbers (determined with Counter) as the key and the current tuple as the value corresponding to that. Once the dictionary construction is completed, we take all the values, which correspond to the tuples.

In the second case, we simply use OrderedDict to maintain the order.

+1 For the the nice combination of OrderedDict, frozenset, and Counter. — Raymond Hettinger, Apr 29 '14 at 06:24

score 1 · Answer 4 · answered Apr 29 '14 at 05:56

1

May be this?:

result = {tuple(sorted(x)) for x in all_the_ways}

answered Apr 29 '14 at 05:56

mshsayem

17,557
11
61
69

score 1 · Answer 5 · answered Apr 29 '14 at 07:25

1

Try

from collections import OrderedDict
print OrderedDict.fromkeys(map(lambda x: tuple(sorted(x)), all_the_ways)).keys()

or

print set(map(lambda x: tuple(sorted(x)), all_the_ways))

answered Apr 29 '14 at 07:25

Kei Minagawa

4,395
3
25
43

shx2 · Answer 6 · 2014-04-29T05:58:06.907

0

I take it that you consider two elements "equal" if they contain the same values, regardless of the order.

So you can "canonicalize" each tuple by sorting it, convert back to tuples (so they are hashable), and remove duplicates using set again:

set( tuple(sorted(tup)) for tup in all_the_ways )

You can also preserve the original "outer" order, by using OrderedSet instead of set.

edited Apr 29 '14 at 05:58

answered Apr 29 '14 at 05:52

shx2

61,779
13
130
153

Get list of unique multi-sets

6 Answers6