0

I have extremely few image data for my experiment. It's only 40 images for each classes in my dataset. So I want to augment all the train, validation, and test sets by using the ImageDataGenerator below:

train_datagen = ImageDataGenerator(
    horizontal_flip=True,
    shear_range=0.2,
    zoom_range=0.2,
    rescale=1/255.,
)
val_datagen = ImageDataGenerator(
    horizontal_flip=True,
    shear_range=0.2,
    zoom_range=0.2,
    rescale=1/255.,
)
test_datagen = ImageDataGenerator(
    horizontal_flip=True,
    shear_range=0.2,
    zoom_range=0.2,
    rescale=1/255.,
)

I used them with flow_from_directory function, use the train and validation generators in model training, but the validation data seems not augmented (the number of images in validation set seems not changed from the initial number). Somehow, this also happens in test set. I found it out when I evaluate the model with sklearn's classification_report. The number displayed in the support column of the result is the exact same as number of images in test set before the augmentation.

                             precision    recall  f1-score   support

                 Blue Whale       1.00      1.00      1.00         4
               Killer Whale       1.00      0.75      0.60         4
                      Shark       0.50      0.75      0.75         4

                   accuracy                           0.90        12
                  macro avg       0.83      0.83      0.78        12
               weighted avg       0.83      0.83      0.78        12

How to use the Tensorflow generator to augment the validation and test sets?

Dhana D.
  • 1,670
  • 3
  • 9
  • 33
  • Augmenting test and validation doesn't really make sense. If you classify augmented data wrong, how can you be sure it's not because of poor quality augmentation? – Julien Jul 27 '21 at 05:54
  • So should I just augment the test and validation sets manually? – Dhana D. Jul 27 '21 at 05:57
  • Jullien has a right point in there but I've seen a couple of approaches that uses augmentation on test and validation set. They do it by augmenting test and validation and then getting mean results from augmentation per image. – Hakan Akgün Jul 27 '21 at 09:53

0 Answers0