Iterating over images in folder blowing up memory

Question

I'm using the following code to iterate over images in a folder and to save them in a dictionary with the filenames as keys, but it's blowing up memory requirements quickly even though the images in the folder should easily fit into memory. Why is this, and how can I fix it?

def make_image_dict(folders):
  image_dict={}
  for folder in folders:
    files = os.listdir(folder)
    print ("Loading images in folder:", folder)
    for file in files:
      try:
        image=ndimage.imread(folder+'/'+file)
        if file.endswith('.png'):
          image_name = str(file)[:-4]
          image_dict[image_name]=image
      except IOError as e:
        print (e)
  return image_dict

png-Images are compressed and uncompressed in memory. Did you consider this? — Daniel, Jan 11 '17 at 22:09

score 0 · Answer 1 · answered Jan 11 '17 at 22:14

You do not want to store the actual files into memory, just the file names. That way you can load and unload the files one by one later in the application. Loading that much data is kinda pointless.

Try This

import os

pictures = []
files = os.listdir('images')
for file in files:
    if (file.endswith(".png")):
        pictures.append(file)

....

for picture in pictures:
    workingPicture = read(picture)
    analyze(picture)
    inspect(picture)
    ...

Iterating over images in folder blowing up memory

1 Answers1