4

How to plot a graph for false positives per window vs miss rate (or probability of false alarm) and ROC (receiver operating curve) for an object detection application using video?How to determine the number of false positives and hits?An example will be very useful.

Shreya M
  • 107
  • 1
  • 2
  • 11
  • Do you have a test set? The false positives is simply items the machine thought were "true" while in fact they are "false". – amit Feb 03 '13 at 08:55

1 Answers1

2

Its pretty simple. Store all your true positive (H0) values in one array and all your true negatives (H1) values in another.

Sort both lists.

Find the highest value from both lists and the lowest value from both lists. Divide the range by an appropriate number (e.g 1000), this will be your step.

Now step from minimum to maximum by the step value.

For each evaluation value find the point in the h0 and h1 array that is greater than that value. Divide this index by the number of values in the h0/h1 array and multiply by 100 (giving you a percentage).

  • False rejection (fr) = h0 index percentage.
  • False acceptance (fa) = 100 - (h1 index percentage).

Plot fa against, 100 - fr.

To calculate the EER you need to find the minimum distance between the fr and fa calculated above.

float diff = fabsf( fr - fa );
if ( diff < minDiff )
{
    minDiff = diff;
    minFr   = fr;
    minFa   = fa;
}

And then at the end the EER is calculated as follows:

float eer = (minFr + minFa) / 2.0f;

Edit: The values you get for H0 and H1 are simply score values indicating the "likelihood" that your match is correct. You must calculate these numbers somewhere as you must make a decision on whether you recognise your object or not based on this score.

The H0 list is the scores you get when you have definite matches. The H1 list is the scores you get when you have definite non-matches.

Goz
  • 61,365
  • 24
  • 124
  • 204
  • Thank you for your reply.But, I have not understood by what you mean "Store all your true positive (H0) values in one array and all your true negatives (H1) values in another.Sort both lists" What kind of numbers are they?Suppose I have 20 images of the object to be detected.Let the number of true matches be 15 and remaining is 5 false matches.Then what does that list contain which needs to be sorted out?Could you also explain this with a code? – Shreya M Feb 03 '13 at 20:08
  • @ShreyaM Well you can't calculate a ROC/DET plot without knowing your true positives and your true negatives ... – Goz Feb 03 '13 at 20:14
  • Generally you have loads more true negatives (H1s) than rue positives (H0s) – Goz Feb 03 '13 at 20:15
  • Also to get a good ROC plot you need way more than 20 tests ... with 20 only you will get a very poor plot... – Goz Feb 03 '13 at 20:16
  • So,again apologize for bothering but could you explain with a small example what actually will be the data for true positive and true negatives..are they determined using a counter say one counts how many matches are there. – Shreya M Feb 03 '13 at 20:32
  • Also,correct me if I am wrong >>1. let total test T set=50,h0=20,h1=30.Should h0+h1=T? 2. highest value in h0=20 and highest value in h1=30 after this I do not understand what needs to be done from your explanation :( – Shreya M Feb 03 '13 at 20:35
  • Thank you,one last query still I have doubts on how to assign the scores..are they probabilistic values?Could you kindly put a sample of putting scores since I have not yet followed. – Shreya M Feb 04 '13 at 20:24
  • @ShreyaM: What do you mean by "Putting scores"? The values are probabilities. Usually they are log likelihoods (as this is what a GMM and other algorithms output) but the scores could be anything. The higher the score the higher the indication that the score represents a win, usually. – Goz Feb 05 '13 at 09:42