How do I solve this problem during transfer learning ResNet-50?

Question

I am training Resnet-50 to classify 9 classes. I am using following code, transfer learning, to train the model.

Train and test loss and accuracy seem to be fine but when I am testing network against new images I see lots of mistakes. I feel like that the model is not learning well, I was wondering if you please let me know what is wrong in my approach? How do I solve this problem?

NUM_CLASSES = 9

CHANNELS = 3

IMAGE_RESIZE = 224
RESNET50_POOLING_AVERAGE = 'avg'
DENSE_LAYER_ACTIVATION = 'softmax'
OBJECTIVE_FUNCTION = 'categorical_crossentropy'

LOSS_METRICS = ['accuracy']

NUM_EPOCHS = 100
EARLY_STOP_PATIENCE = 3


STEPS_PER_EPOCH_TRAINING = 10
STEPS_PER_EPOCH_VALIDATION = 10


BATCH_SIZE_TRAINING = 100
BATCH_SIZE_VALIDATION = 100

BATCH_SIZE_TESTING = 1

from tensorflow.python.keras.applications import ResNet50
from tensorflow.python.keras.models import Sequential
from tensorflow.python.keras.layers import Dense



resnet_weights_path = '/path/resnet50_weights_tf_dim_ordering_tf_kernels_notop.h5'
model = Sequential()

model.add(ResNet50(include_top = False, pooling = RESNET50_POOLING_AVERAGE, weights = resnet_weights_path))

model.add(Dense(NUM_CLASSES, activation = DENSE_LAYER_ACTIVATION))

model.layers[0].trainable = False

from tensorflow.python.keras import optimizers

sgd = optimizers.SGD(lr = 0.01, decay = 1e-6, momentum = 0.9, nesterov = True)
model.compile(optimizer = sgd, loss = OBJECTIVE_FUNCTION, metrics = LOSS_METRICS)


model.summary()
from tensorflow.python.keras import optimizers

sgd = optimizers.SGD(lr = 0.01, decay = 1e-6, momentum = 0.9, nesterov = True)
model.compile(optimizer = sgd, loss = OBJECTIVE_FUNCTION, metrics = LOSS_METRICS)


from keras.applications.resnet50 import preprocess_input
from keras.preprocessing.image import ImageDataGenerator

image_size = IMAGE_RESIZE


data_generator = ImageDataGenerator(preprocessing_function=preprocess_input)


train_generator = data_generator.flow_from_directory(
        '/path train folder/train',
        target_size=(image_size, image_size),
        batch_size=BATCH_SIZE_TRAINING,
        class_mode='categorical')

validation_generator = data_generator.flow_from_directory(
        '/path test folder/test',
        target_size=(image_size, image_size),
        batch_size=BATCH_SIZE_VALIDATION,
        class_mode='categorical')


(BATCH_SIZE_TRAINING, len(train_generator), BATCH_SIZE_VALIDATION, len(validation_generator))


from tensorflow.python.keras.callbacks import EarlyStopping, ModelCheckpoint

cb_early_stopper = EarlyStopping(monitor = 'val_loss', patience = EARLY_STOP_PATIENCE)
cb_checkpointer = ModelCheckpoint(filepath = '/path/best.hdf5', monitor = 'val_loss', save_best_only = True, mode = 'auto')




fit_history = model.fit_generator(
        train_generator,
        steps_per_epoch=STEPS_PER_EPOCH_TRAINING,
        epochs = NUM_EPOCHS,
        validation_data=validation_generator,
        validation_steps=STEPS_PER_EPOCH_VALIDATION,
        callbacks=[cb_checkpointer, cb_early_stopper]
)
model.load_weights("/path/best.hdf5")

model.save('transfer_resnet.h5')


print(fit_history.history.keys())

I think you could reduce the code by half by removing the unnecessary parts. That would make it simpler for us to help. Also, some critical parts are missing, e.g., the part where you make predictions. — Nicolas Gervais, Jul 02 '20 at 13:09

score 0 · Answer 1 · answered Jul 02 '20 at 13:06

0

This can have many reasons. For one, it is possible that your data set is too small or not varied enough. What you can try is to add a few more Dense Layers in the top section.

answered Jul 02 '20 at 13:06

NielsSchneider

181
3
6

My dataset is not small, @Nicolas I edited my post. – nikki Jul 02 '20 at 16:48

How do I solve this problem during transfer learning ResNet-50?

1 Answers1