2

I implemented a FCN network to do semantic segmentation. I am using Cityscapes as my dataset. As you know, there are some classes in Cityscapes that you ignore during the training and it is labeled as 255. I used weighted loss to ignore the loss for the unknown classes(set the loss to zero for unknown class). Now I want to exclude unknown class from my evaluation metric(mean Intersection Over Union (mIOU)).It is not clear for me how to exclude the unknown class at this point.

At the moment I am considering all the classes including the unknown class like this using tensorflow method:

 miou, confusion_mat = tf.metrics.mean_iou(labels=annotation, predictions=pred_annotation, num_classes=num_cls)

with tf.control_dependencies([tf.identity(confusion_mat)]):
    miou = tf.identity(miou)

I tried this , but it give an error for unbound label(for the unkonwn label)

miou, confusion_mat = tf.metrics.mean_iou(labels=annotation, predictions=pred_annotation, num_classes=(num_cls-1))
Shai
  • 111,146
  • 38
  • 238
  • 371
Arb
  • 91
  • 10
  • I am not familiar with the Cityscapes dataset, but why would you ignore some classes during training? Then the network cannot learn these classes, no? You would split your dataset in train and test, but use all classes, ...? – anki Mar 11 '19 at 14:54
  • Thanks for your comment anki. Well the point is to learn every classes except unknown class. The unknown class is defined for any pixel of the input image which is out of interesting classes, or some objects which are far from camera and not obvious for doing an correct class annotation. Since the unknown class can be resulted from different roots as I mentioned, I do not want to have it in my training. – Arb Mar 11 '19 at 17:02

1 Answers1

0

If you have a class that you want to ignore during the mIoU calculation, and you have access to the confusion matrix then you can do it like this:

  1. ignore the miou calculated by tensorflow (since it considers all classes and that is not what you want)
  2. remove row and column from the confusion matrix that correspond to the class you want to ignore
  3. recalculate miou metric with the new confusion matrix

How to recalculate miou metric from the confusion matrix?

  • iou for the first class: iou_0 = conf_mat[0,0] / (sum(conf_mat[0,:]) + sum(conf_mat[:,0]) - conf_mat[0,0])
  • iou for the second class: iou_1 = conf_mat[1,1] / (sum(conf_mat[1,:]) + sum(conf_mat[:,1]) - conf_mat[1,1])
  • ...
  • in general for the class j: iou_j = conf_matrix[j,j] / (sum(conf_mat[j,:]) + sum(conf_mat[:,j]) - conf_mat[j,j])

At the end, sum and average all these per class iou to get miou.

cijad
  • 152
  • 3