What does this image-kernel correlation matrix represent?

Question

I am experimenting with some computer vision techniques, specifically feature detection. I am trying to identify features by conducting auto-correlation between an image and a feature-kernel.

However, the resulting correlation-matrix doesn't make sense to me... Can anyone help me understand how to interpret or visualize this matrix, so that it's apparent where the feature is located?

Feature Kernel:

Original Image:

Code:

import cv2
import pprint
import numpy
import scipy.ndimage
from matplotlib import pyplot as plt
import skimage.feature

# load the image
img = cv2.imread('./lenna.jpg')[:,:,0]
f_kernel = cv2.imread('./lenna_feature.jpg')[:,:,0]

def matched_filter(img, f_kernel, detect_thres):
    result = scipy.ndimage.correlate(img, f_kernel)
    print("Feature Match Template")
    plt.imshow(skimage.feature.match_template(img, f_kernel))
    plt.show()
    return result

plt.imshow(matched_filter(img,f_kernel,1))
print("Correlation Matrix")
plt.show()

Result:

So, in the first result-image, there's an obvious maximum-point at (150,200). I am interpreting this as the most likely location of the feature.

However, in the second result-image, correlation matrix result, there is no obvious pattern. I was expecting that there would be some, obvious high-correlation point.

Help?

Your correlation result might look different if you cast your image and template to a floating-point type first. I think we're looking at the result of integer overflow here. — Cris Luengo, Apr 27 '20 at 23:42

score 2 · Accepted Answer · answered Apr 27 '20 at 23:41

skimage.feature.match_template computes the normalized cross correlation. That is, for each location over the image, the image patch and the template are normalized (subtract mean and divide by standard deviation) and then multiplied together and averaged. This computes the correlation coefficient of the image patch and the template. The correlation coefficient is a value between 1 and -1. A correlation coefficient of 1 indicates that the image patch is a linear modification of the template (i.e. constant1 * template + constant2).

scipy.ndimage.correlate computes the correlation (same as convolution, but without mirroring the kernel). That is, here we do not first normalize the image patch. Places where the image has higher values will automatically also have a higher correlation, even if not at all similar to the template.

yar · Answer 2 · 2020-04-27T23:05:15.107

1

I can't reproduce your result. For me, the second image has a shape that looks a bit like Lena.

Anyway, you don't want to use the correlation, you want to use the correlation coefficient to do template matching. Pure correlation is not normalized, so it's rather an averaging filter than a template matching.

Edit: Added the corellation image

edited Apr 27 '20 at 23:05

answered Apr 27 '20 at 21:29

yar

1,855
13
26

Thank you. If you have a moment, could you share your resulting image? – Matthew E. Miller Apr 27 '20 at 21:51
1

Here you go. I added the corellation image – yar Apr 27 '20 at 23:06
Thank you so much, that's very interesting. – Matthew E. Miller Apr 28 '20 at 01:52

What does this image-kernel correlation matrix represent?

2 Answers2