How to correctly train VGG16 Keras

Question

I'm trying to retrain VGG16 to classify Lego images. However, my model has a low accuracy (between 20%). What am I doing wrong? Maybe the number of FC is wrong, or my ImageDataGenerator. I have approx. 2k images per class and a total of 6 classes.

How I create the model:

def vgg16Model(self,image_shape,num_classes):
    model_VGG16 = VGG16(include_top = False, weights = None)
    model_input = Input(shape = image_shape, name = 'input_layer')
    output_VGG16_conv = model_VGG16(model_input)
    #Init of FC layers
    x = Flatten(name='flatten')(output_VGG16_conv)
    x = Dense(256, activation = 'relu', name = 'fc1')(x)
    output_layer = Dense(num_classes,activation='softmax',name='output_layer')(x)
    vgg16 = Model(inputs = model_input, outputs = output_layer)
    vgg16.compile(loss = 'categorical_crossentropy', optimizer = 'adam', metrics = ['accuracy'])
    vgg16.summary()
    return vgg16

I'm creating ImageDataGenerator and training:

path = "real_Legos_images/trainable_classes"
evaluate_path = "real_Legos_images/evaluation"
NN = NeuralNetwork()
gen = ImageDataGenerator(rotation_range=40, width_shift_range=0.02, shear_range=0.02,height_shift_range=0.02, horizontal_flip=True, fill_mode='nearest')

train_generator = gen.flow_from_directory(os.path.abspath(os.path.join(path)), 
                target_size = (224,224), color_mode = "rgb", batch_size = 16, class_mode='categorical')

validation_generator = gen.flow_from_directory(os.path.abspath(os.path.join(evaluate_path)),
                target_size = (224,224), color_mode = "rgb", batch_size = 16, class_mode='categorical')

STEP_SIZE_TRAIN = train_generator.n//train_generator.batch_size
num_classes = len(os.listdir(os.path.abspath(os.path.join(path))))
VGG16 = NN.vgg16Model((224, 224, 3), num_classes)

VGG16.save_weights('weights.h5')
VGG16.fit_generator(train_generator, validation_data = validation_generator, validation_steps = validation_generator.n//validation_generator.batch_size,
                steps_per_epoch = STEP_SIZE_TRAIN, epochs = 50)

score 0 · Answer 1 · answered Apr 14 '19 at 01:27

The VGG16 model with the parameter include_top = False will return 512 dimensions feature maps. Usually, we should add a GlobalAveragePooling2D or GlobalMaxPooling2D layer after it first, then flat it to an one-dimensional array. Otherwise, you will get an array which is too long to fit.

score 0 · Answer 2 · answered Apr 15 '19 at 15:13

0

You have set the weight property to 'None' for VGG which means your networks is initialized with random weights. This means you are not using the pre-trained weights. So, I would suggest to try setting the weights to 'imagenet' such that you can use the VGG networks that its weights are pretrained on imagenet dataset:

model_VGG16 = VGG16(include_top=False, weights='imagenet')

answered Apr 15 '19 at 15:13

Rane

41
1
5

That was intentional, I want to train the net into a whole different domain dataset, and I really don't need the pre-trained weights – Raphael Cardoso Fernandes Apr 15 '19 at 20:15
Thank you for the comment. I see. But, I would suggest to set the weights to imagenet and set the trainable of the layers taken from VGG16 to false. Classify your data with the features extracted by such network and see if you get any better accuracy. VGG may still give excellent accuracy on completely different image datasets. – Rane Apr 15 '19 at 21:26

How to correctly train VGG16 Keras

2 Answers2