Python Generator call counter not increasing unless yield is called

Question

I am using the scandir library to look over all files on my computer. I would like to stop the loop once I've checked 100 records. If the yield statement is never called, my variable, c never increases and the loop doesn't stop. I've put in some bogus file name thisfilewontbefound that will never be found thus yield is never reached. Why is c not incrementing?

from scandir import scandir, walk
import sys

def subdirs(path):
    for path, folders, files in walk(path):
        for files in scandir(path):
            if 'thisfilewontbefound' in files.path:
                yield files.path

c = 0
for i in subdirs('C:\\'):
    if c > 100:
        print "test over"
        sys.exit()
    c += 1
    print i

Because `subdirs` never yields, the body of `for i in subdirs(...)` never runs. — gil, Feb 29 '16 at 18:35
@user2242044 the first iteration of subdirs will never yield any elements if the file does not exist. — James Buck, Feb 29 '16 at 18:37
`subdirs` will still iterate but without `yield`-ing anything to iterate through for the `for` loop, so at the end `for` loop never runs — Iron Fist, Feb 29 '16 at 18:38

score 1 · Accepted Answer · answered Feb 29 '16 at 18:39

1

Your for loop is waiting for the generator. The generator will only produce a value if yield is executed. But yield is never executed because there is no such file, so the for loop will wait a long time, until all files on your C: drive have been scanned and the generator ends without ever yielding anything.

Either put the counter in the generator, or don't filter on files in your generator and have it yield more often.

answered Feb 29 '16 at 18:39

Martijn Pieters

1,048,767
296
4,058
3,343

for your second suggestion of having it yield more often, would it best best to let the generator do the filtering or the for loop in terms of efficiency? – user2242044 Feb 29 '16 at 18:53
@user2242044: without the filter, your generator is nothing more than an extra empty wrapper around the `os.walk()` generator. There is not really any point in having that generator at that point. :-) – Martijn Pieters Feb 29 '16 at 19:04

Python Generator call counter not increasing unless yield is called

1 Answers1