VGG Transfer learning - predict generator shows different result than evaluate generator

Question

I built a CNN based on VGG pre-trained model. I fine tuned the last conv block (#5) together with my added Fully connected layers. My classification problem has 4 classes, my last activation layer is 'softmax' and I am using 'sparse_categorical_crossentropy' as my loss function. When I am creating my train/valid generator, I am using class_mode ('sparse'). So I fit my model and I get 92% accuracy on my validation dataset after 2 epochs. The problem that I have, is when I check the predict_generator to see my predictions on my validation set the accuracy is around 73 True predictions/ (73 + 161 wrong predictions) = 0.31%, however the model.evaluate_generator(validation_generator) has 91% accuracy.

train_datagen = ImageDataGenerator(rescale=1./255)

test_datagen = ImageDataGenerator(rescale=1./255)
train_generator = train_datagen.flow_from_directory(
        path_data_train,
        target_size=(img_width, img_height),
        batch_size=16,
        class_mode="sparse")

validation_generator = test_datagen.flow_from_directory(
        path_data_valid,
        target_size=(img_width, img_height),
        batch_size=16,
        class_mode="sparse")

nb_train_samples = 1874
nb_validation_samples = 234
epochs = 10
batch_size = 16

history = model.fit_generator(
        train_generator,
        steps_per_epoch=nb_train_samples // batch_size,
        epochs=epochs,
        validation_data=validation_generator,
        validation_steps=nb_validation_samples // batch_size,
        callbacks = [checkpointer])

Epoch 1/10
117/117 [==============================] - 42s 356ms/step - loss: 0.0850 - acc: 0.9690 - val_loss: 0.4173 - val_acc: 0.9062
Epoch 2/10
117/117 [==============================] - 42s 360ms/step - loss: 0.0690 - acc: 0.9765 - val_loss: 0.4423 - val_acc: 0.894

print (model.metrics_names)
model.evaluate_generator(validation_generator)
['loss', 'acc']
[0.39189313988909763, 0.9059829049640231]

preds = model.predict_generator(validation_generator)

Any help is appreciated.

score 0 · Answer 1 · answered Oct 26 '18 at 13:27

I post the answer if in case someone in future hit the same issue. You need to set Shuffle = False in validation_generator.

validation_generator = test_datagen.flow_from_directory(
        path_data_valid,
        target_size=(img_width, img_height),
        batch_size=16,
        class_mode="sparse", shuffle = False)

VGG Transfer learning - predict generator shows different result than evaluate generator

1 Answers1