1

I'm trying to use 'inception resnet v2.py' to do a multi-label classification.

I used sigmoid, but the result is not good.

Do you know exactly where to change?

https://github.com/tensorflow/models/tree/master/research/slim

"train_image_classifier.py" has been changed to sigmoid, but results are as good as using softmax.

Do I have to change it from "inception resnet v2.py" in the "net" folder?

  if 'AuxLogits' in end_points:
    slim.losses.sigmoid_cross_entropy(
        end_points['AuxLogits'], labels,
        label_smoothing=FLAGS.label_smoothing, weights=0.4,
        scope='aux_loss')
  slim.losses.sigmoid_cross_entropy(
      logits, labels, label_smoothing=FLAGS.label_smoothing, weights=1.0)
  return end_points

If put in a color red and 4 wheel drive car image, It would be nice if it came out like this, but it is not.

Car [0.99]
4 wheel drive [0.99]
color red [0.99]

In reality, everyone guessed right, but it comes out as if using softmax.

Car [0.99]
4 wheel drive [0.03]
color red [0.009]
AZDDKKS
  • 41
  • 1
  • 8

1 Answers1

2

There are some possible choices depending on the type of your "multi-label".

If the possible overlapping part of your "multi-label" is combinations of different mutually independent label sets(color, shape, etc), then creating softmax output layers for these sets separately will be good.

If unluckily labels can't be divided, then you may need to check and change the loss function: for softmax it's usually cross entropy, which does not work well for sigmoid.

HzCheng
  • 401
  • 4
  • 11
  • It was changed to sigmaid because it was not independent of each other, but the result was the same as softmax. Can I know which part I have to modify? – AZDDKKS May 07 '19 at 04:49
  • @AZDDKKS 1. According to your example, "was not independent of each other" does not mean it can't be divided into different subsets ,can you show us the complete label list? 2. I just roughly go through the code and yes, it seems the definition of output layer is in line 475 of https://github.com/tensorflow/models/blob/master/research/slim/nets/inception_v2.py and only changing loss function is not enough. – HzCheng May 07 '19 at 05:00
  • 1. I simply follow the example. The folder label structure is changed to tfrecord through 'down_load_and_convert.py' For example ~/car_photos/red_colors/photos1.jpg ~/car_photos/red_colors/photos2.jpg ... ~/car_photos/4 wheel drive/photos3.jpg ~/car_photos/4 wheel drive/photos4.jpg ... ~/car_photos/Rear-wheel drive/photos5.jpg ~/car_photos/Rear-wheel drive/photos6.jpg 2. I didn't just fix the loss, I changed it to sigmaid for the output layer you said, but it's still like softmax – AZDDKKS May 07 '19 at 05:23
  • And I'm not 'inception v2'. I'm using 'inception resnet v2'. – AZDDKKS May 07 '19 at 05:33
  • I faced similar issue before and my solution is to find the minimal element label set containing n labels and then train n predictors simultaneously with one output layer. Each predictor only predict one single label and the combination of all predicted labels is the result. Or you can try classifier chains. – HzCheng May 08 '19 at 02:15
  • Thank you. I'll find out what you told me. I have one more question. Is there a problem if I put the same image in each folder in the dataset structure, where the folder name is labeled on the image, and then go ahead with classification learning? For example, there are two-wheel drive, four-wheel drive, car color, and car type. This refers to the overlapping of four-wheel drive red sports car images in each label folder. – AZDDKKS May 09 '19 at 02:26
  • It should be ok but what I suggest more is using a dict like data structure rather than playing with the folder structure. – HzCheng May 09 '19 at 03:03