1

I am wondering about the loop behavior when I am popping each element but that could apply to any modifications on an iterrables.

Let's imagine this:

l = ["elem1", "elem2", "elem3", "elem4"]
for i, elem in enumerate(l):
    l.pop(i)

It is simple but i am wondering: does python keep a unmodified instance of l at each loop iteration or does it update l ? I could loop over a l.copy() but in that case i have an IndexError.

I know that i could simply solve the issue of removing elem in list one by one but here i am trying to understand the behavior.

pacdev
  • 551
  • 1
  • 3
  • 13
  • 5
    What happened when you tried? Did you *inspect* what python does, e.g. by ``print(i, elem, l)`` inside the loop? – MisterMiyagi Dec 16 '21 at 09:20
  • 1
    Does this answer your question? "[Strange result when removing item from a list while iterating over it](https://stackoverflow.com/q/6260089/90527)", "[Modifying list while iterating](https://stackoverflow.com/q/1637807/90527)" – outis Dec 16 '21 at 09:20
  • It is unspecified by the standard... Said differently anything can happen and only the programmer will be to blame. – Serge Ballesta Dec 16 '21 at 09:21
  • Yes i tested it but was not sure how to interpret the print results. Thanks for the post sharing, it helps. actually i understand that i shoudl never alter a list I am looping over. – pacdev Dec 16 '21 at 13:14

3 Answers3

4
for i in iterable: 
    # some code with i

is (with sufficient precision in this context) equivalent to

iterator = iter(iterable)
while True:
    try:
        i = next(iterator)            
    except StopIteration:
        break
    # some code with i

You can see

  • i is reassigned in each iteration
  • the iterator is created exactly once
  • mutations to the iterable may or may not lead to unexpected behavior, depending on how iterable.__iter__ is implemented. __iter__ is the method responsible for creating the iterator.

In the case of lists the iterator keeps track of an integer index, i.e. which element to pull next from the list. When you remove an item during iteration, the index of the subsequent elements change by -1, but the iterator is not being informed of this.

>>> l = ['a', 'b', 'c', 'd']
>>> li = iter(l) # iterator pulls index 0 next
>>> next(li)
'a' # iterator pulled index 0, will pull index 1 next
>>> l.remove('a') # 'b' is now at index 0
>>> l
['b', 'c', 'd']
>>> next(li)
'c'
timgeb
  • 76,762
  • 20
  • 123
  • 145
1

Why don't you just test it?

>>> for i, elem in enumerate(l):
...     print(i, l)
...     l.pop(i)
...
0 ['elem1', 'elem2', 'elem3', 'elem4']
'elem1'
1 ['elem2', 'elem3', 'elem4']
'elem3'

Under the hood it just calls the next() method on the iterable (PEP279). Because the list, indeed, changes on each iteration, it just raises StopIteration as the index would reach 2 in your case.

Lodinn
  • 462
  • 2
  • 9
1

When everything else fails, read the manual!

Note There is a subtlety when the sequence is being modified by the loop (this can only occur for mutable sequences, e.g. lists). An internal counter is used to keep track of which item is used next, and this is incremented on each iteration. When this counter has reached the length of the sequence the loop terminates. This means that if the suite deletes the current (or a previous) item from the sequence, the next item will be skipped (since it gets the index of the current item which has already been treated). Likewise, if the suite inserts an item in the sequence before the current item, the current item will be treated again the next time through the loop. This can lead to nasty bugs that can be avoided by making a temporary copy using a slice of the whole sequence, e.g., [...]

I'm not sure the bit about a "counter" that gets "incremented" is strictly true, but the gist is there: don't modify and iterate!

Ture Pålsson
  • 6,088
  • 2
  • 12
  • 15