Training accuracy reaches 1.0 after the first training epoch

Question

I am working on a transfer learning model (based off of Vgg19) with the following architecture.

_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
input_1 (InputLayer)         (None, 224, 224, 3)       0
_________________________________________________________________
block1_conv1 (Conv2D)        (None, 224, 224, 64)      1792
_________________________________________________________________
block1_conv2 (Conv2D)        (None, 224, 224, 64)      36928
_________________________________________________________________
block1_pool (MaxPooling2D)   (None, 112, 112, 64)      0
_________________________________________________________________
block2_conv1 (Conv2D)        (None, 112, 112, 128)     73856
_________________________________________________________________
block2_conv2 (Conv2D)        (None, 112, 112, 128)     147584
_________________________________________________________________
block2_pool (MaxPooling2D)   (None, 56, 56, 128)       0
_________________________________________________________________
block3_conv1 (Conv2D)        (None, 56, 56, 256)       295168
_________________________________________________________________
block3_conv2 (Conv2D)        (None, 56, 56, 256)       590080
_________________________________________________________________
block3_conv3 (Conv2D)        (None, 56, 56, 256)       590080
_________________________________________________________________
block3_conv4 (Conv2D)        (None, 56, 56, 256)       590080
_________________________________________________________________
block3_pool (MaxPooling2D)   (None, 28, 28, 256)       0
_________________________________________________________________
block4_conv1 (Conv2D)        (None, 28, 28, 512)       1180160
_________________________________________________________________
block4_conv2 (Conv2D)        (None, 28, 28, 512)       2359808
_________________________________________________________________
block4_conv3 (Conv2D)        (None, 28, 28, 512)       2359808
_________________________________________________________________
block4_conv4 (Conv2D)        (None, 28, 28, 512)       2359808
_________________________________________________________________
block4_pool (MaxPooling2D)   (None, 14, 14, 512)       0
_________________________________________________________________
block5_conv1 (Conv2D)        (None, 14, 14, 512)       2359808
_________________________________________________________________
block5_conv2 (Conv2D)        (None, 14, 14, 512)       2359808
_________________________________________________________________
block5_conv3 (Conv2D)        (None, 14, 14, 512)       2359808
_________________________________________________________________
block5_conv4 (Conv2D)        (None, 14, 14, 512)       2359808
_________________________________________________________________
block5_pool (MaxPooling2D)   (None, 7, 7, 512)         0
_________________________________________________________________
flatten_1 (Flatten)          (None, 25088)             0
_________________________________________________________________
dense_1 (Dense)              (None, 4096)              102764544
_________________________________________________________________
dropout_1 (Dropout)          (None, 4096)              0
_________________________________________________________________
dense_2 (Dense)              (None, 1024)              4195328
_________________________________________________________________
dropout_2 (Dropout)          (None, 1024)              0
_________________________________________________________________
dense_3 (Dense)              (None, 512)               524800
_________________________________________________________________
dense_4 (Dense)              (None, 2)                 1026
=================================================================

Problem: training error results (shown below) reflect meaningful values in the first epoch. As soon as the model reaches the second epoch accuracy reaches 1.0 - which is impossible. This behavior does not change when I switch VGG to Inception, when I add regularization, when I switch between different optimizers (sgd, addagrad, rmsprop), or when I switch between losses (categorical_crossentropy, mean_squared_error).

Also, the classification results for all test/val images is [[1. 0. 0. 0. 0.]] meaning the classifier tends to favor class0 always.

Epoch 1/2
10/10 [==============================] - 109s 11s/step - loss: 1.7893 - acc: 0.9000

Epoch 00001: val_acc improved from -inf to 0.60000, saving model to vgg19_12.h5
Epoch 2/2
10/10 [==============================] - 122s 12s/step - loss: 0.9368 - acc: 1.0000

Ask: Any thoughts what might be the root-cause of this problem?

This seems like overfitting, is your dataset tiny? And/or not mostly the same class? — Aaron N. Brock, Apr 23 '18 at 17:42
Perhaps you made an error during preprocessing and all of your outputs ended up as the same class — Primusa, Apr 23 '18 at 22:06
Hi @Milad M were you able to get a solution for this? I am facing the same error. Could you please give some inputs. Thanks — prb_cm, Sep 03 '20 at 09:11

Training accuracy reaches 1.0 after the first training epoch

0 Answers0