Python for loop implementation

Question

Can someone tell me how exactly Python's for loops are implemented? The reason I'm asking this is because I'm getting different behavior in the following two for loops when I expect the same behavior (assuming cases is just a set of elements):

First for loop:

for case in cases:
    blah

Second for loop:

for i in range(len(cases)):
    case = cases[i]
    blah

I'm running my code in a multi-threaded environment.

Basically, I'm wondering whether Python's for loop's iterating over a set (as in the first for loop) is simply a quickhand way of the second one. What exactly happens when we use the python for loop, and is there any underlying optimization/ implementation that may be causing the behavior difference I'm observing?

If `cases` is actually a `set`, the second snippet doesn't work. Does it happen to be a `list`? — , May 05 '13 at 18:10
Two comments: (1) What do you mean by multi-threaded environment. Have you created multiple threads in your Python code? If that's the code, please share the code for the same. (2) You can possibly put prints in your for loops and try to debug. You require to put more information on what exactly is the behaviour say you try to print the value of case in every iteration. — Vishal, May 05 '13 at 18:10
What is the different behavior you are seeing? Can you show an example that demonstrates the different behavior? — BrenBarn, May 05 '13 at 18:43
@vishal I am using Scrapy, which to my knowledge uses multi-threading. I considered putting prints to debug but I'd like to understand how for loops in python work exactly too so I put up this question :D — Penguinator, May 05 '13 at 18:43
Actually I would like to know what difference you see in the output.. Otherwise knowing the answer to your question might not necessarily solve your issue. — Vishal, May 05 '13 at 19:15

Martijn Pieters · Accepted Answer · 2013-05-05T18:30:31.943

17

No, the second format is quite different.

The for loop calls iter() on the to-loop-over sequence, and uses next() calls on the result. Consider it the equivalent of:

iterable = iter(cases):
while True:
    try:
        case = next(iterable)
    except StopIteration:
        break

    # blah

The result of calling iter() on a list is a list iterator object:

>>> iter([])
<list_iterator object at 0x10fcc6a90>

This object keeps a reference to the original list and keeps track of the index it is at. That index starts at 0 and increments until it the list has been iterated over fully.

Different objects can return different iterators with different behaviours. With threading mixed in, you could end up replacing cases with something else, but the iterator would still reference the old sequence.

edited May 05 '13 at 18:30

answered May 05 '13 at 18:11

Martijn Pieters

1,048,767
296
4,058
3,343

There's a weird exception: An object with a `__getitem__` which accepts integers in `range(0, n)` and throws `IndexError` for parameter n can be iterated without a `__iter__` method. I don't think any builtin types actually do this though. – May 05 '13 at 18:18
1

could you clarify how this would cause him to get different outputs? – Drew Verlee May 05 '13 at 18:18
1

In addition, the second forces python to generate a large list. – James May 05 '13 at 18:41
I see. I suspect the difference might be in the iterator for the object I'm using then. That makes a lot of sense, thanks for the straightforward answer! – Penguinator May 05 '13 at 18:53
@delnan when you wish to iterate over something that only has `__getitem__`, the the command `iter(some_string)` will return a generic iterator that simply handles the `IndexError`, so essentially the exception will still be `StopIteration` – Guy Jul 17 '16 at 04:30

score 0 · Answer 2 · answered Mar 05 '20 at 05:53

0

l = [1, 2, 3, 4, 5]
l = iter(l)
while True:
    try:
        print l.next()
    except StopIteration:
        exit()

answered Mar 05 '20 at 05:53

Satyajit Das

37
5

score -2 · Answer 3 · answered May 05 '13 at 18:25

-2

i didn't get any difference,check this below, is it what u exactly trying..

>>> cases = [1,2,3]
>>> for case in cases:
...     print case
...
1
2
3
>>> i=0
>>> for i in range(len(cases)):
...     print cases[i]
...
1
2
3
>>>

answered May 05 '13 at 18:25

user2049919

79
1
1
4

Python for loop implementation

3 Answers3