-1

For a performance measuring purpose I am trying to draw ROC curve. In ROC curve I have to plot False Positive Rate (FPR) in x-axis and True Positive Rate (TPR) in y-axis. As we know,

FPR = FP/(FP+TN)

So in the following picture how can i detect True Negative(TN) ? I have used HOG classifier to detect human. I marked with rectangle 1,2,3,4,5,6(or should be 7) to show the human objects that should be ignored and not to classify as human. and I think those are True Negative.

True Negative detetion in this frame

In this picture i want to say my assumption,as we know,

False negative: Result should have been positive, but is negative.

False positive: Result should have been negative, but is positive.

True positive: Result should have been positive and is positive.

True negative: Result should have been negative and is negative

So i think in this frame FP = 0, TP = 0, FN = 0 but not sure about TN, is it 6 or 7 or anything other? Please correct me also about FP, TP, and FN if i am wrong. I saw this question How to categorize True Negatives in sliding window object detection? which was really helpful but still i have to calculate FPR for this scenario.

Community
  • 1
  • 1
Md. Nahiduzzaman Rose
  • 442
  • 2
  • 9
  • 23
  • but what does this image represent? What are this rectangles with numbers? Is this rectangle a place where your algorithm shouted "this is a human"? If so - what is "underneth" is there anything human like? In order to use classificaation langauge you also need to show **every single place** where your algorithm said "nothing interesting", and this will be probably HUGE number (since in sliding window you have like thousands of "negative" answers). – lejlot Sep 22 '16 at 18:15
  • Yes those rectangle and numbers are "This is human". It is a frame from a running video, where this situation occurs, I am calculating FP,TP,TN,FN in every frame. So in this frame is TN = 7 ? "since in sliding window you have like thousands of "negative" answers" So how can i calculate FPR and draw ROC curve? @lejlot – Md. Nahiduzzaman Rose Sep 22 '16 at 18:19
  • This sounds off-topic, likely more on topic in stats.SE – catastrophic-failure Sep 27 '16 at 12:19

1 Answers1

1

You cannot calculate these values from such image, you need more data (knowledge what is actually happening). But what you need is probably just total amount of these windows, which is some constant N. Now, it seems like all these windows are wrong (none is on the human), thus:

  • FP = 6 (your method claims there are 6 people, but none of these claims is valid since they are completely off - however if this is just visualization issue, and method actually captured valid people, this 6 should be moved to TP instead)
  • TP = 0 (it does not correctly mark any human)
  • FN = 10 (if I counted correctly there are 10 people at this image, and all these are missing)
  • TN = N - 16, where N is number of all analized windows, since all of them are correctly classified as "lack of human" up to 10 FNs and 6 FPs, which add up to these 16.

In general

  • FP = how many actual not humans are marked "human"
  • TP = how many actual humans are marked "human"
  • FN = how many actual humans are correctly ignored (not marked "human")
  • TN = how many actual not humans are correctly ignored (not marked "human")
lejlot
  • 64,777
  • 8
  • 131
  • 164
  • Well HOG method provides a Rectangle(detection window) to detect the human, In this frame i think no human should be detected as some are so tiny, and incomplete. So there is no detection by the algorithm and it is correct, So I think FP = 0, as there is no detection of an incomplete human object, TP =0, as no rectangle no human, FN = 0, no false rectangle or false detection and TN = 10 (If there is 10 human like object as you count) for only this frame as no detectable object(human) exist and correctly ignored. I am sorry i am just saying my understanding and confidence – Md. Nahiduzzaman Rose Sep 22 '16 at 18:40
  • This is completely not how it works, of course you can define what resolution of "being human" is important for you, but you cannot arbitrarly change definition of these terms. It returned 6 frames, none of them is human (with high or low resolution), so FP is 6, period. There is no discussion here. FN is either 0 or 10, depending on your resolution. TN is not 10, as it depends on how many frames were actually analyzed. This is how the term is defined. However, as a meta problem - if there are **no humans**, you cannot define these objects at all, it is not a correct classification problem – lejlot Sep 22 '16 at 18:47
  • Your approach would mean that you have some preconditioning, which preselected "human-like objects" and the algorithm is supposed to just correctly classify which "human-like" is actual human or not. This is fine, but you need to actually **have** this method (in script, not as a human). Do you have it? This is highly unlikely case, as it requires you to have a perfect "human-like" preselector and position of "detected humans" already show this is **not** how it works - it clearly selects "as a human" things which are not even human-like, so it is not a subset of your "10 human-like things" – lejlot Sep 22 '16 at 18:49
  • http://stackoverflow.com/questions/16271603/how-to-categorize-true-negatives-in-sliding-window-object-detection in this question's image FP = 2 isn't it? As there are two falsely red rectangle searching for human, But in my image you are not having something like these FALSE RECTANGLE. FN =1 as 1 rect saying, and TP =3 as rect saying. That is how i thought. – Md. Nahiduzzaman Rose Sep 22 '16 at 18:55
  • Yes, FP=2 for the separate question. And in image your provided there are **6** rectangles, which incorrectly say that there is a human. Look at my answer, I am stating that FP=6 all the time. Unless you treat them as ok, since they are "a bit off but close enough", this is very domain specific, but I would say is invalid from image recognition perspective. – lejlot Sep 22 '16 at 18:58
  • Please dont be angry, Oh So sorry I said before .Those rectangle are not from the algorithm generated , I marked this manually!!! as you count 10 I count 6 and marked it with rectangle to the image – Md. Nahiduzzaman Rose Sep 22 '16 at 19:03
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/123979/discussion-between-nahiduzzaman-rose-and-lejlot). – Md. Nahiduzzaman Rose Sep 22 '16 at 19:05
  • So your algorithm did not produce **anything**, and rectangles are your **human annotations**? Then this is a huge difference, and it will be FP=0, TP=0, FN=6, TN=N-6 (where N is defined as before) – lejlot Sep 22 '16 at 19:07
  • I marked(with rectangle) those to describe my understanding on True Negative. Well Sir, You said FN = 6, because of my "marking" , but I am also neglecting those 6 human, I think those should not be detected by the HOG and counting them as TN. so HOG Algorithm is correct here and correctly ignored those objects. So FN = 0 – Md. Nahiduzzaman Rose Sep 22 '16 at 19:18
  • I am also not clear about N (N is number of all analized windows) , is analized window means analized frame? So I am just exploring the current frame, so analized window N = 1 in my case? – Md. Nahiduzzaman Rose Sep 22 '16 at 19:24
  • You seem to change meaning from answer to answer, now you are telling that "what I marked is neither human, nor the answer of the model, there are no humans here, and model also thinks so". This has nothing to do with previous answers, and in such case you would have FP=0, TP=0, FN=0, TN=N, where N is number of **windows** analyzed, and no, N is not 1, N is **how many windows were subject to analysis during HOG execution**. Thus if you have a sliding window of size 4x4 on image with 64x64 it is roughly ((64-4+1)^2 = 3721). Of course the problem is there are windows with "almost a human"... – lejlot Sep 22 '16 at 21:17