how to print out each class's Average Precision using Mxnet Faster RCNN for object detection

Question

I am using faster rcnn(mxnet) for object detection on my own dataset, which has 9 classes(including background). However, i found that in the end it only prints out the average accuracy over all 9 classes during trainning process. Furthermore, during test process, it also only prints out the average precision and recall over all 9 classes. I am wondering how can i print out each class's accuracy during the training process, and each class's recall and precision during the test process? Or can someone tell me where should i look at to approach my goal? An ideal example will be shown in the image. enter image description here

Answer 1

You can use the Scikit-learn function sklearn.metrics.precision_recall_fscore_support¶ for this. And sklearn.metrics.classification_report for a prettified version.

At test time, you will have an array of true values (Y_true) and an array of predicted probabilities for each class (Y_prob). Use these as follows;

Y_pred = np.argmax(Y_prob, axis=1)
print(classification_report(Y_true, Y_pred))

         precision    recall  f1-score   support
class 0       0.50      1.00      0.67         1
class 1       0.00      0.00      0.00         1
class 2       1.00      0.67      0.80         3
avg / total   0.70      0.60      0.61         5

Slightly more work is required for these to be listed every N batches at training time. You can set a callback argument, and a custom eval_metric if you're using module.fit method;

model = mx.mod.Module(symbol=...)
model.fit(..., batch_end_callback = mx.callback.Speedometer(batch_size),
          eval_metric=custom_metric, ...)

You'll need to create a new class for custom_metric that extends mxnet.metric.EvalMetric and implements a get method that prints out (or even returns) the per class metrics.