0

I have a list of files I get from doing a pathlib.glob(). From that list, I need to work with them in chunks depending on their extension (so first .1, then .2, etc.).

I can only get the first group of files to show up, and after that I keep getting empty lists. I know i can workround this issue by just changing my initial glob, but the fact that I'm not seeing the issue means I don't understand how something works under the hood, and I'd like some help figuring it out.

This is my first attempt using list comprehension:

Import sys, os
from pathlib import Path
folder = Path(os.getcwd())
files = folder.glob('*dat*')
for i in range(4):
    extension = '*.'+str(i+1)
    cat = [x for x in files if x.match(extension)]
    print(cat)

prints [files with .1] [] [] []

and I get the same output using filter

for i in range(4):
        extension = '.'+str(i+1)
        cat = list(filter(lambda x:str(x).endswith(extension), files))

Clearly I'm missing something but i dont know what.

Mauro
  • 93
  • 4
  • So you basically want to filter out all the files with same extension, right? @Mauro –  May 21 '21 at 16:30
  • Yes, i needed to split them into different lists so i could work with them in batches, depending on their extension. I used a dictionary for this, but i had issues iterating over the files object. Solution is pinned. – Mauro May 22 '21 at 18:11

1 Answers1

1

It seems to me that it happens because files is a generator. So in the list comprehension at your first iteration of for loop, its contents are already consumed, and so at the subsequent iterations there is no more items left!

So you might want to make a list out of files first: try putting

files = list(files)

just before the for loop.

j1-lee
  • 13,764
  • 3
  • 14
  • 26