-1

How can I calculate the false positive rate for an object detection algorithm, where I can have multiple objects per image?

In my data, a given image may have many objects. I am counting a predicted box as a true positive if its IOU with a truth box is above a certain threshold, and as a false positive otherwise. For example: I have 2 prediction bounding boxes and 2 ground-truth bounding boxes:

I computed IoU for each pair of prediction and ground-truth bounding boxes: IoU = 0.00, 0.60, 0.10, 0.05 threshold = 0.50

In this case do I have TP example or not? Could You explain it?

Badum
  • 70
  • 1
  • 1
  • 10

2 Answers2

0

Summary, specific: Yes, you have a TP; you also have a FP and a FN.

Summary, detailed: Your prediction model correctly identified one GT (ground truth) box. It missed the other. It incorrectly identified a third box.

Classification logic:

At the very least, your IoU figures should be a matrix, not a linear sequence. For M predictions and N GT boxes, you will have a NxM matrix. Your looks like this:

0.00 0.60
0.10 0.05

Now, find the largest value in the matrix, 0.60. This is above the threshold, so you declare the match and eliminate both that prediction and that GT box from the matrix. This leaves you with a rather boring matrix:

0.10

Since this value is below the threshold, you are out of matches. You have one prediction and one GT remaining. With the one "hit", you have three objects in your classification set: two expected objects, and a third created by the predictor. You code your gt and pred lists like this:

gt   = [1, 1, 0]    // The first two objects are valid; the third is a phantom.
pred = [1, 0, 1]    // Identified one actual box and the phantom.

Is that clear enough?

Prune
  • 76,765
  • 14
  • 60
  • 81
0

You can use an algorithm (e.g. Hungarian algorithm aka Kuhn–Munkres algorithm aka Munkres algorithm) to assign detections to ground truths. You might incorporate the ability to not assign a detection to ground truth & vice versa (e.g. allow for false alarms and missed detections).

After assigning the detections to ground truths, just use the definition of TPR Wikipedia page for Sensitivity (aka TPR) & Specificity (aka TNR)

I provide this answer since I think @Prune provided an answer which uses a Greedy algorithm to perform assignment of detections to ground truths (i.e. "Now, find the largest value in the matrix, 0.60. This is above the threshold, so you declare the match and eliminate both that prediction and that GT box from the matrix."). This Greedy assignment method will not work well in all scenarios. For example imagine a matrix of IoU values between detections and ground truth bounding boxes

      det1 det2
pred1 0.4  0.0
pred2 0.6  0.4

The Greedy algorithm would assign pred2 to det1 and pred1 to det2 (or pred1 to nothing if accounting for possibility of false alarms). However, the Hungarian algorithm would assign pred1 to det1 and pred2 to det2, which might be better in some cases.

user3731622
  • 4,844
  • 8
  • 45
  • 84