2

i have a problem to plot a curve for miss rate vs false positive rate to analyze the performance of my proposed system (as sampled on picture below). I have two samples dataset for positive and negative sample. I want to plot the performance of my system whether it can classify people or non people with this curve.

As far as I know, I need to get True Positive and False Positive values after the classification, but I am not sure yet how to plot the curve. Any one can help please??

Sample of curve

Indrasyach
  • 147
  • 3
  • 17
  • What are you plotting against? Miss rate against false positive (one on each axis), both against some other variable (some setting of your system), etc... – nkjt Dec 16 '14 at 11:13
  • I want to plot the curve as illustrated on picture above, miss rate against false positive per image to analyze the performance of my system. But i have no idea to scratch it. I want to compare my system with two other methods. @nkjt – Indrasyach Dec 17 '14 at 02:31
  • If you've got those two bits of data, just `semilogx` to get the log on the x like in the image. Is *plotting* or *calculating* the data your issue? – nkjt Dec 17 '14 at 11:14
  • my problem concerns on both issues. I dont know how to obtain the computation. Could you explain me how to do the calculation of data? I have been looking through some literatures but found nothing. @nkjt – Indrasyach Dec 18 '14 at 01:41
  • You need some "ground truth" (e.g. testing image set, such as INRIA ). Miss rate = percentage of people (or whatever you're looking for) which aren't found. "False positives" = number of 'people' found who aren't people, per image. As you increase the sensitivity/reduce thresholds for a "hit", you usually get less misses but more false positives. – nkjt Dec 18 '14 at 11:06
  • I am using also INRIA dataset. I am detecting on head-shoulder of people. For example, now I am testing the system using INRIA dataset, positive samples (90x160) and my detection window is 32x32, after classified, in one image will be detected 2 boxes, one is the correct one another is not correct. How do I calculate Miss rate and False positive rate from that result? could you provide me more details on the calculation, maybe useful links? I am really appreciate for it. @nkjt – Indrasyach Dec 18 '14 at 12:06

3 Answers3

1

Since MATLAB R2017a, you can use evaluateDetectionMissRate function.

[logAverageMissRate,fppi,missRate] = evaluateDetectionMissRate(detectionResults,groundTruthData)

This function returns data points for plotting the log MR-FPPI curve. (MR: Miss-Rate, FPPI: False Positive Per Image).

For an example of its usage, type doc evaluateDetectionMissRate command in MATLAB or go to here.

Cris Luengo
  • 55,762
  • 10
  • 62
  • 120
ashkan
  • 476
  • 6
  • 14
0

There are two type of bounding boxes in the object detection, the boxes that data-set labeled theme as object and second the boxes that your algorithm detects.

If your bbox have huge intersection with data-set bbox, it is okey.

If your bbox have NOT intersection with data-set bbox it is False Posative.

And we call All data-set bbox without intersection with your bbox in an image MISS Rate. and after calculating these numbers, plotting these values is straight forward.

malloc
  • 604
  • 10
  • 18
-1

You can use the following GitHub repo for plotting MR vs FPPI. It may seem like the code only calculates the mAP but it does much more than that. It also calculates the miss-rate, false positives per image and the log-average miss rate. All these are computed in the main.py file present in the repo(line 81) but are not plotted. All you have to do is just plot the MR vs FPPI using matplotlib (or any other module). Just follow the ReadMe file to get started. Hope this helps!

Jitesh Malipeddi
  • 2,150
  • 3
  • 17
  • 37