2

I was working on the Normalized Cross Correlation for Template Matching in Spatial domain. While the method is slow, it works good enough for my purpose. But I saw a weird thing in there. Let me explain the situation below:

91 91                        91 91 9 9
91 91                        91 91 9 9
                                   8   6 7 8

pattern image          source image.

Now, when NCC goes through this: It finds the mean of the template image as 91 and underlying source image also as 91 and then it subtracts the intensity value from the pixel which essentially takes all terms in the formula to zero resulting in an undefined correlation value and no matches are found even when there is a perfect match.! How to get around this situation? I am using the following formula: from the excellent source by J. P. Lewis

enter image description here

Also, when I modified the formula to subtract (mean/2) from every pixel intensity, it seemed to work fine but I am concerned as to how much vulnerable to Illumination this new correlation coefficient is.

Edit: Conditions even worsened when I took a 1 X 1 pattern image and had multiple occurrences in source image. Using the above modified version I was unable to find appropriate matches. I would love to look into various work arounds many of you might have been using. Thanks.!

Community
  • 1
  • 1
peterd
  • 99
  • 9

1 Answers1

2

The idea of the normalized cross correlation is that the similarity doesn't change if you add an arbitrary number to every pixel or multiply every pixel by an arbitrary (non-negative) number. Now take any 2x2 pixel area in the search image, e.g. 

91  9
 6  7

Multiply this by 0 and add 91 - and you have a perfect match. So in a nutshell: You can't match a "flat" template using normalized cross-correlation. Or a "flat" area in the search image, either.

Note that this isn't a "bug" in the normalized cross correlation. The effect you're seeing makes perfect sense. Imagine someone hands you a completely black photograph and asks you what you see. Your answer wouldn't be "I have perfect match for the batmobile, because it's completely black", your answer would be "I can't tell, there's too little contrast in the image". Which is precisely what the NCC is trying to tell you with the division by zero.

Also, when I modified the formula to subtract (mean/2) from every pixel intensity

You mean, you replaced the mean's in the numerator and denominator by mean/2? That doesn't sound like a good idea. If the template or search image area contains only zeros, you will still get a division by zero. More importantly: you're calculating a quantity that has no real meaning (at least none that I can think of). For example, the average brightness of the search area region will influence on the matching result.

I would love to look into various work arounds [...]

The obvious ad-hoc workaround would be to add a small quantity to the denominator, so a "flat" area in the search image won't lead to devision by zero. Then you get (more or less) a similarity measure that doesn't change if you add an arbitrary number to every pixel or multiply every pixel by an arbitrary (non-negative) number unless that number is very close to 0.

But that would give you a 0 match for any flat search area region or flat template. Which (as explained above) makes perfect sense. If you want different behavior in this case, you don't want a normalized cross correlation.

An alternative similarity measure might be squared euclidean distance. You can optionally subtract the mean of the template and search area before calculating the difference. Then you get a similarity measure that doesn't change if you add an arbitrary number to every pixel. But it will change if you multiply every pixel by some value.

Niki
  • 15,662
  • 5
  • 48
  • 74
  • @nikie... thanks for your response. The things you mentioned did help me understand the behavior. While I agree that contrast was not enough which led NCC to show no results. But what in case of 1X1 template image; We can't talk of contrast in that since the contrast ratio is always 1. Does it mean that NCC cannot be used in this case? Extending it: I think NCC won't be helpful in case of any template image/source image area where contrast ratio = 1. – peterd Mar 12 '13 at 02:32
  • Also, I am now thinking of putting a check in my program that if contrast ratio is 1 then use a different similarity measure instead of NCC. Does that sound clean? – peterd Mar 12 '13 at 02:35
  • @peterd: Exactly, NCC is undefined if there is no contrast in the template or in the respective area in the search image. And it makes no sense to talk about "correlation" with a 1x1 template. – Niki Mar 12 '13 at 07:45
  • And I'm not sure about using two different similarity measures. If you think `91, 91, 91` is a perfect match for the template `91, 91, 91`, but not for e.g. `70, 70, 70`, then you probably don't want NCC at all. It also might have strange effects if your program gets completely different results for the templates `1, 1, 1` and `1, 1, 1.0001` because it uses a completely different similarity for the former. – Niki Mar 12 '13 at 07:49