itertools.groupby() values disappear

Question

I have a nested list [['A','B'],['A','C'],['D'],['C','A']]

import itertools
d = {}
for key, group in itertools.groupby([['A','B'],['A','C'],['D'],['C','A']], lambda x: x[0]):
    d[key] = list(group)
print(d)

Now I will have {'A': [['A', 'B'], ['A', 'C']], 'D': [['D']], 'C': [['C', 'A']]}

However, since 'D' is the only nested list with a single element, I want all single-element to print out as a number 0.

I want to output: {'A': [['A', 'B'], ['A', 'C']], 'D': 0, 'C': [['C', 'A']]}, so I added an if-else clause and changed my code to:

for key, group in itertools.groupby([['A','B'],['A','C'],['D'],['C','A']], lambda x: x[0]):
    if len(list(group)[0]) > 1:
        d[key] = list(group)
    else: d[key] = 0
print(d)

However, the above code printed out as {'A': [], 'D': 0, 'C': []}. I didn't see anything wrong with my code. However, why do A and C print out an empty list?

Your first `list` exhausts the generator. you need assign to a temp variable. — dawg, Jul 06 '21 at 18:30
i.e. put `group = list(group)` in the loop, then just use `group` in the `if` block. — Barmar, Jul 06 '21 at 18:31
Thank you! Creating a new variable works, but I still don't quite understand why it doesn't work if I don't create a new variable... — Troy Bear, Jul 06 '21 at 18:34
Because `group` is not a list; it's an iterator that can only be used once. — chepner, Jul 06 '21 at 18:35
but if I can only use it once, at least it should print out: {'A': [['A', 'B'], ['A', 'C']], 'D': 0, 'C': []}, I mean 'A' should have its elements. Why it print out all [] empty lists for both A and C? Where did I use the generator the first time? — Troy Bear, Jul 06 '21 at 18:46
@TroyBear *Where did I use the generator the first time?* When you call `len(list(group)[0])` — dawg, Jul 06 '21 at 19:03

dawg · Answer 1 · 2021-07-06T21:28:29.237

The problem you are seeing is the first call of list(group) inside of the call to len exhausts the iterator that group points to; you can only use it once. You therefor need to save the values in a temp list.

You can do:

import itertools 

d = {}
li=[['A','B'],['A','C'],['D'],['C','A']]

for key, group in itertools.groupby(li, lambda x: x[0]):
    gv=list(group)
    if len(gv) > 1:
        d[key] = gv
    else: d[key] = 0
    
print(d)
# {'A': [['A', 'B'], ['A', 'C']], 'D': 0, 'C': 0}

You can also shorten a bit with:

for key, group in itertools.groupby(li, lambda x: x[0]):
    gv=list(group)
    d[key]=gv if len(gv) > 1 else 0

You asked Where did I use the generator the first time?

Consider the following example:

>>> gen=(x*2 for x in (1,2,3))  # a generator expression
>>> type(gen)
<class 'generator'>
>>> list(gen)
[2, 4, 6]
>>> list(gen)
[]

If you use list on a generator, that exhausts the generator. You used list(group) inside of len which exhausts the generator. With your second call to list -- there is nothing left.

but if I can only use it once, at least it should print out: {'A': [['A', 'B'], ['A', 'C']], 'D': 0, 'C': []}, I mean 'A' should have its elements. Why it print out all [] empty lists for both A and C? Where did I use the generator the first time? — Troy Bear, Jul 06 '21 at 18:46

score 1 · Answer 2 · answered Jul 06 '21 at 18:33

1

The problem you are facing is that when you call list(group) you are forcing the generator to generate all elements, so next time you use it it has no elements on it.

Use the following:

import itertools

d = {}

for key, group in itertools.groupby([['A','B'],['A','C'],['D'],['C','A']], lambda x: x[0]):
    group_list = list(group)
    d[key] = group_list if len(group_list[0]) > 1 else 0
print(d)

Output

{'A': [['A', 'B'], ['A', 'C']], 'D': 0, 'C': [['C', 'A']]}

answered Jul 06 '21 at 18:33

lmiguelvargasf

63,191
45
217
228

but if I can only use it once, at least it should print out: {'A': [['A', 'B'], ['A', 'C']], 'D': 0, 'C': []}, I mean 'A' should have its elements. Why it print out all [] empty lists for both A and C? Where did I use the generator the first time? – Troy Bear Jul 06 '21 at 18:46
@TroyBear, I am not sure what you are asking. When you convert a generator into a list the generator has no more elements to use. Try the following if this helps to understand: `>>> generator = (1 for _ in range(5))` then, `>>> list(generator)` and then again `>>> list(generator)`. The first time you call `list(generator)` you will get a list of all elements, but the generator will no longer have elements, so next time you will get an empty list. – lmiguelvargasf Jul 06 '21 at 18:53

itertools.groupby() values disappear

2 Answers2