1

I have a program where I get one image as input and I have to compare it with ~640 known images to see which one is the most similar. To do this I was thinking of using OpenCV's match template, as it seems very fast and effective in doing what I want to do.

I noticed that matching two images, both 400x240 px, 1000 times is much slower than matching a 400x240 px image in a 1400x240 px, despite both of them being 1000 matches. My idea was to combine the 640 images in one big image containing them in a grid (easy to do since they all have the same size).

Doing this I could really speed up the process if I could match template the input image in only some of the subimages of the big combined reference (the ones with the top-left corner in a "grid pixel", as in a subimage that was actually one of those 640 images that I used to combine into the big reference image), but it doesn't seem to be there a way to tell match template only to match on a specified set of positions.

How could I go to speed up this process? Is there a different library (I'm working with Python) that does something similar to match template but where I can specify which subimages to match? Is there an entirely different approach that suits more my goal?

EDIT: Basically what I have to do is: I'm taking a screenshot from a videogame and in the screenshot there's a clean portrait of some character (400x240 px). I have clean portraits of all the characters (80 of them, 8 skins each for a total of 640 portraits) and I want to find which portraits is the closest to the one in the screenshot so that I can identify the character played. It'd be awesome if I could also "mask" some pixels like with matchTemplate from OpenCV as there are some specific parts of the 400x240px rectangle that change from player to player and I'd like if I didn't have to account for that and could just mask out those (known) pixels

Fran314
  • 31
  • 3
  • 1
    Use cuda opencv to increase the performance. – Hihikomori Dec 20 '20 at 16:02
  • Thank you! I'll look it up! Though I was wondering if there was a way to speed it up by approaching the solution more efficiently rather than just increasing the brute force power, as I'm planning to run this piece of code on computers that don't necessarely have a strong GPU – Fran314 Dec 20 '20 at 16:20
  • 1
    GPU will be faster even it is poor. It is more effective on GPU because of parallel computing approach. – Hihikomori Dec 20 '20 at 17:27
  • 1
    do NOT stack/tile your pictures. do NOT use matchTemplate for this, if you aren't looking for the *position* of the match but only need to know how well one picture matches another when overlaid precisely. use `absdiff` (OpenCV) and sum it, or square and sum it. that's the so-called "Sum of Squared Errors" – Christoph Rackwitz Dec 20 '20 at 23:53
  • 1
    it would help immensely if you explained the problem/goal on a very high level, without your proposed solution/approach. also please post some sample data (pictures) and clarify if you need the position of best match or not. matchTemplate is *expensive*, so you should avoid that if it's *not* the right tool for the job (and that remains to be determined) – Christoph Rackwitz Dec 20 '20 at 23:55
  • @ChristophRackwitz I've edited the question to add the high level explanation. Also, I've tried the sum of absdiff way and it is indeed way faster. I've also tried the sum of squared errors and it's a bit slower than the absdiff (understandibly, given the squaring required). Is there any reason why one should choose the latter? Anyway, thanks! – Fran314 Dec 21 '20 at 00:37
  • Also, the NOT tiling the pictures refers to the idea to using matchTemplate or is it a general thing? As in, would it be a bad idea in general to save all the images in one big image and then compare the image from the screenshot to subimages picked from a grid? 'Cause saving all the images in one big image would make the resources folder a lot cleaner, but if there are drawbacks I won't go that way – Fran314 Dec 21 '20 at 00:40
  • 1
    statistics, regression, least squares (!) fit... commonly assume that errors should be "weighted" like that. it also makes things differentiable (|x| can't be differentiated near 0, x² can). you can stick with sum of absolute differences. that's a common enough measure. – Christoph Rackwitz Dec 21 '20 at 01:43
  • 1
    I take it then you already know precisely at which location in the screenshot to look for that portrait? in that case, matchtemplate makes very little sense. the function was made for finding *where* things match, not only to judge how well two pictures match. it does that by testing *every* possible location, shifting pixel by pixel. that would be several orders of magnitude more work (also pointless work) compared to simply calculating sum of absolute differences for ~640 templates. – Christoph Rackwitz Dec 21 '20 at 01:46
  • 1
    sure, you can save a big mosaic, if you like one file instead of 640. that one file is going to be 400*240*640 = 61.44 megapixels large. that vs 640 individual pictures makes no difference in RAM. I would find that a bit unwieldy though. the mosaic also doesn't give you any benefit for the comparison/lookup part. you'd still take the mosaic apart, comparing the screenshot's region with every one of 640 templates in turn. – Christoph Rackwitz Dec 21 '20 at 01:49

0 Answers0