0

I am working on loop-closure detection problem in two different seasons, e.g., summer, and fall. I need to make precision-recall curves. Suppose, I have taken 500 image from summer and 500 image from fall season. I have distance matrix. enter image description here

But I am totally confused, how to make precison recall curves. Like, for each image from one season, I will get 500 nearest images in ascending (distance) order. I know the definition of precision and recall, but i can't get close to the solution of this problem. Looking forward for any kind of help or comments or advice. thanks in advance.

Hrid Biswas
  • 1
  • 3

1 Answers1

0

In precision-recall plots each point is a pair of precision and recall values. In your case, I guess, you'd need to compute those values for each image and then average them.

Imagine you have 1000 images in total and only 100 images that belong to summer. If you take 500 closest images to some "summer" image, precision in the best case (when the first images always belong to the class) would be:

precision(summer) = 100 / (100 + 400) = (retrieved summer images) / (retrieved summer images + other retrieved images) = 0.2

And recall:

recall(summer) = 100 / (100 + 0) = (retrieved summer images) / (retrieved summer images + not retrieved summer images) = 1

As you can see, it has high recall because all the summer images were retrieved, but low precision, because there are only 100 images, and 400 other images don't belong to the class.

Now, if you take the first 100 images instead of 500, both recall and precision would equal 1.

If you take 50 first images, then precision would be still 1, but recall would drop to 0.5.

So, by varying the number of images you can get points for the precision-recall curve. For the above-described example these points would be (0.2, 1), (1, 1), (1, 0.5).

You could compute these values for each of the 1000 images using different thresholds.

lawful_neutral
  • 633
  • 8
  • 29
  • I really appreciate your help. Thank you. – Hrid Biswas Sep 24 '18 at 00:21
  • @HridBiswas I just edited the comment a bit, because I got the question wrong. Anyway, I hope I didn't reply too late :) – lawful_neutral Sep 24 '18 at 00:22
  • actually my problem is slightly diferent than the way you have described, but still it is very helpful for me and i can modify your solution in my way. But i have one problem that is: If i take precision vs recall values from your example i.e., (0.2, 1), (1, 1), (1, 0.5), then i see, for recall 1, I have 2 pricision values, o.2 and 1 (since (0.2, 1), (1, 1)), then how can I plot it? thanks in advance – Hrid Biswas Sep 26 '18 at 18:13
  • @HridBiswas if you google "precision-recall curves", you could see that a lot of plots have straight lines, i.e. there are two or more values of recall for the same precision (and the other way around). https://classeval.files.wordpress.com/2015/06/perfect-precision-recall-curve1.png?w=420&h=224 for instance, this picture shows perfect precision and recall, and precision equals 1 as recall changes from 0 to 1. – lawful_neutral Sep 26 '18 at 23:41