What do I do to improve my Keras CNN VGG16 model

Question

I'm working in a project that has 700 images for 2 classes (1400 total). I'm using VGG16 but i'm new with this model and I don't know what could I do to improve this model..

This is my model:

vgg16_model = VGG16(weights="imagenet", include_top=True)

# (1) visualize layers
print("VGG16 model layers")
for i, layer in enumerate(vgg16_model.layers):
    print(i, layer.name, layer.output_shape)

# (2) remove the top layer
base_model = Model(input=vgg16_model.input, 
                   output=vgg16_model.get_layer("block5_pool").output)

# (3) attach a new top layer
base_out = base_model.output
base_out = Reshape((25088,))(base_out)
top_fc1 = Dense(256, activation="relu")(base_out)
top_fc1 = Dropout(0.5)(top_fc1)
# output layer: (None, 5)
top_preds = Dense(1, activation="sigmoid")(top_fc1)

# (4) freeze weights until the last but one convolution layer (block4_pool)
for layer in base_model.layers[0:14]:
    layer.trainable = False

# (5) create new hybrid model
model = Model(input=base_model.input, output=top_preds)

# (6) compile and train the model
sgd = SGD(lr=1e-4, momentum=0.9)
model.compile(optimizer=sgd, loss="binary_crossentropy", metrics=["accuracy"])

history = model.fit([data], [labels], nb_epoch=NUM_EPOCHS, 
                    batch_size=BATCH_SIZE, validation_split=0.1)

# evaluate final model
vlabels = model.predict(np.array(valid))

model.save('model.h5')

... that gives me the follow return:

Train on 1260 samples, validate on 140 samples
Epoch 1/5
1260/1260 [==============================] - 437s 347ms/step - loss: 0.2200 - acc: 0.9746 - val_loss: 2.4432e-05 - val_acc: 1.0000
Epoch 2/5
1260/1260 [==============================] - 456s 362ms/step - loss: 0.0090 - acc: 0.9984 - val_loss: 1.5452e-04 - val_acc: 1.0000
Epoch 3/5
1260/1260 [==============================] - 438s 347ms/step - loss: 1.3702e-07 - acc: 1.0000 - val_loss: 8.4489e-05 - val_acc: 1.0000
Epoch 4/5
1260/1260 [==============================] - 446s 354ms/step - loss: 4.2592e-06 - acc: 1.0000 - val_loss: 7.6768e-05 - val_acc: 1.0000
Epoch 5/5
1260/1260 [==============================] - 457s 363ms/step - loss: 0.0017 - acc: 0.9992 - val_loss: 1.1921e-07 - val_acc: 1.0000

It seems to be a bit overfitting..

My predict.py:

def fix_layer0(filename, batch_input_shape, dtype):
    with h5py.File(filename, 'r+') as f:
        model_config = json.loads(f.attrs['model_config'].decode('utf-8'))
        layer0 = model_config['config']['layers'][0]['config']
        layer0['batch_input_shape'] = batch_input_shape
        layer0['dtype'] = dtype
        f.attrs['model_config'] = json.dumps(model_config).encode('utf-8')

fix_layer0('model.h5', [None, 224, 224, 3], 'float32')

model = load_model('model.h5')

for filename in os.listdir(r'v/'):
    if filename.endswith(".jpg") or filename.endswith(".ppm") or filename.endswith(".jpeg") or filename.endswith(".png"):
        ImageCV = cv2.resize(cv2.imread(os.path.join(TEST_DIR) + filename), (224,224))
        ImageCV = cv2.addWeighted(ImageCV,4, cv2.GaussianBlur(ImageCV,(0,0), 224/25), -4, 120) #The same process made when I get data in train
        ImageCV = ImageCV.reshape(-1,224,224,3)
        print(model.predict(ImageCV))

And the results are strange because only the 2 first images are of 'class 0'.. the others are 'class 1':

[[0.99905235]]
[[0.]]
[[1.]]
[[0.012198]]
[[0.]]
[[1.]]
[[1.6363418e-07]]
[[0.99997246]]
[[0.00433112]]
[[0.9996668]]
[[1.]]
[[6.183685e-08]]

What can I do to improve it? I'm a little confused..

score 2 · Answer 1 · edited Sep 30 '19 at 12:14

ImageCV = cv2.addWeighted(ImageCV,4, cv2.GaussianBlur(ImageCV,(0,0),
                          224/25), -4, 120)

Not sure why you do this for the test data. For the validation/test data, usually only normalization is done. During training as well you need to apply the same normalization as a final step before feeding the data to the network.

Refer to this example for fine-tuning VGG16 for a two class problem(dogs vs cats) https://gist.github.com/fchollet/7eb39b44eb9e16e59632d25fb3119975

https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html

To reduce overfitting, you can do data augmentation for the training data i.e. feed the original data & augmented data(apply operations like flip, zoom etc.). Keras ImageDataGenerators make it easy to do the augmentation. Explored in the above tutorial as well.

https://keras.io/preprocessing/image/

I've tried to use data augmentation but still doesn't work.. The acc has been .99 ~ 1. with very low loss (~.0) and the predictions are really bad — , Sep 28 '19 at 13:13

score 1 · Accepted Answer · answered Sep 27 '19 at 18:33

1

First of all, Keras predict will return the scores of the regression (probabilities for each class) and predict_classes will return the most likely class of your prediction. For example, if you classify between cats and dogs, predict could output 0.2 for cat and 0.8 for dog. So, if you use predict, there should be two values per picture, one for each class.

The reason why you only have one value is that your network only has one output neuron. It should have two, as there are two classes.

top_preds = Dense(2, activation="sigmoid")(top_fc1)

If you now want to see most likely class, not the probabilities, you should use predict_classes.

answered Sep 27 '19 at 18:33

Julia K

397
3
10

But in my case any pred > 0.5 is of class 1 and < 0.5 is class 0, right? – Sep 28 '19 at 17:31
No. The pred > 0.5 is for the most likely class. Keras outputs two probabilities. The first one is for class 0, the second one is for class 1. Like this: `[0.4, 0.6]`. You can get the class by running `np.argmax([0.4, 0.6])`. This will output 1 because the index 1 and thus class 1 has greater probability. – Julia K Sep 28 '19 at 17:34
How could I use the predict_classes? _AttributeError: 'Model' object has no attribute 'predict_classes'_ – Sep 29 '19 at 00:22
My mistake. The predict_classes method is only available for the Keras Sequential class but not for the Model class (the class you used for your network). With the Model class, you have to use the `predict` method which will give you a vector of probabilities and then get the `argmax` of this vector (with `np.argmax(vlabels,axis=1)`). – Julia K Sep 29 '19 at 09:27

What do I do to improve my Keras CNN VGG16 model

2 Answers2