2

I am building a multilabel image classification network. The dataset contains 70k images, total number of classes are 12. With respect to the entire dataset, 12 classes has more than 10% images. Out of 12 classes, 3 classes are above 70%. I am using VGG16 network without its associated classifier.

As the training results, I am getting max of 68% validation accuracy. I have tried changing the number of units per Dense layer (512,256,128 etc), increased the number of layers (5, 6 layers), added/removed Dropout layer (with 0.5), kernel_regularization (L1=0.1, L2=0.1).

As accuracy is not the appropriate metric for multilabel classification, I am trying to incorporate HammingLoss as the metric. But it is not working, here is the issue that I opened on the GitHub repo of HammingLoss.

  1. What can be done to improve the accuracy?
  2. What point I am missing in case of incorporating HammingLoss?

For classification, I am using the network as:

network.add(vggBase)
network.add(tf.keras.layers.Dense(256, activation='relu'))
network.add(tf.keras.layers.Dense(64, activation='relu'))
network.add(tf.keras.layers.Dense(12, activation='sigmoid'))


network.compile(optimizer=tf keras.optimizers.Adam(learning_rate=0.001), loss=tf.keras.losses.BinaryCrossentropy(), metrics=['accuracy'])
Shamindra Parui
  • 56
  • 5

1 Answers1

1

I recommend you to use Keras Tuner for tuning.

If Hammingloss is not working for you, you could use a differnet metric as a workaround, like pr_auc for instance. The metric choice depends strongly on what you want to achieve with your model. Maybe towardsdatascience/evaluating-multi-label-classifiers can help you to find that out.

Viktor
  • 583
  • 1
  • 3
  • 10
  • 1
    Okay. I went through the Keras Tuner and the article you mentioned. Other than that, I have removed the a number of data points from the classes that are of above 70% and that only contain single class-label (not more than 1 label). Along with that modification, I have incorporated HammingLoss as discussed in [this](https://stackoverflow.com/questions/65356172/how-to-implement-hamming-loss-as-a-custom-metric-in-keras-model) this post. As the result, I am getting 16% of validation HammingLoss. – Shamindra Parui Feb 21 '23 at 10:37
  • @ShamindraParui is your question answered then? Btw it is always good to use a smaller baseline model to have something to compare your results with. Also, I recommend to test your model on entirely independet test data. – Viktor Feb 21 '23 at 13:05
  • Hey Viktor, yes it it answered, thanks. I am testing on completely isolated test data. – Shamindra Parui Feb 28 '23 at 12:37