How many images are generated by keras fit_generator?

Question

I use keras to do image augmentation and segmentation. I want to investigate the number of images generated, so I test the following setting of arguments: (1) set batch_size as 1 in flow_from_directory when define the generator:

def myGene(...):
    ...
    image_datagen = ImageDataGenerator(**aug_dict)
    image_generator = image_datagen.flow_from_directory(...,batch_size = 1,..., save_prefix  = 'view',...)
    mask_datagen = ImageDataGenerator(**aug_dict)
    mask_generator = mask_datagen.flow_from_directory(...,batch_size = 1,..., save_prefix  = 'view',...)
    ...

(2) When training, I set epochs = 1 and steps_per_epoch=1:

model.fit_generator(myGene,steps_per_epoch=1,epochs=1,...)

After training finished, I expect only 1 image and 1 mask are in the directory './view', but I actually found 11 pairs there.

What is wrong here? Did I fail to setting some arguments, or did I do something wrong?

score 1 · Accepted Answer · answered Dec 10 '18 at 17:46

In the fit_generator, there is a hidden argument called max_queue_size=10, which means the generator will try to produce a queue of 10 batches ahead of the current training batch.

You're just seeing the images generated to populate the queue. You can try to use max_queue_size=0, but even though you might see an extra image.

How many images are generated by keras fit_generator?

1 Answers1