simple 1D example showing convolution vs correlation to test commutative/associative property

Question

So I've known this fact that convolution has commutative/associative property while correlation does not but never really written out a simple example to test this out. (proofs in equation forms exist out there)

I was just trying this simple example:

Image : [1 0 -1 0 1 0 1 1]
A kernel = [-1 0 1]
B kernel = [-1 0 1]

Using convolution, I * (A * B) should equal (I * A) * B, but when I tried this they were not equal.. which confused me a lot. would anyone be able to help out? maybe I made a human error in doing computation.. but they should be equal right? (assuming borders are treated as 0s)

And using correlation, the same should not be equal as I understand.. which they dont, but then, my convolution did not either so lol (but it should!)

Any help would be appreciated.

also, A*B (convolution) would be [0 -2 0] right? and AxB (cross correlation) would be [0 2 0]?

score 0 · Answer 1 · answered Aug 29 '20 at 21:27

The convolution operator is commutative. So, you are right that I*(A*B) should be equal to (I*A)*B. Let's convert this to matrix formation first. Convolution by kernel A can be translated to multiplication by the following convolution matrix, C:

C=
 [-1  0  1  0  0  0  0  0]
 [ 0 -1  0  1  0  0  0  0]
 [ 0  0 -1  0  1  0  0  0]
 [ 0  0  0 -1  0  1  0  0]
 [ 0  0  0  0 -1  0  1  0]
 [ 0  0  0  0  0 -1  0  1]
 [ 1  0  0  0  0  0 -1  0]
 [ 0  1  0  0  0  0  0 -1]

We have taken the kernel [-1 0 1] and zero-padded it by 5 zeros on first row. Then, each row below is a cyclic shift by 1 position to the right of the preceding row. So, if we want to compute the convolution of kernel A with I:

I = [1 0 -1 0 1 0 1 1]

We simply compute the matrix-vector multiplication:

C*I^T

(where I^T is the transposed vector).

Following the above formulation, we obtain:

I*(A*B) = (C*C)*I^T

Computing C*C you get the following matrix:

C^2=
 [ 1  0 -2  0  1  0  0  0]
 [ 0  1  0 -2  0  1  0  0]
 [ 0  0  1  0 -2  0  1  0]
 [ 0  0  0  1  0 -2  0  1]
 [ 1  0  0  0  1  0 -2  0]
 [ 0  1  0  0  0  1  0 -2]
 [-2  0  1  0  0  0  1  0]
 [ 0 -2  0  1  0  0  0  1]

and:

I*(A*B) = (C*C)*I^T = 
 [ 4]
 [ 0]
 [-2]
 [ 1]
 [ 0]
 [-2]
 [-2]
 [ 1]

whereas (I*A)*B is in matrix formulation C*(C*I^T) which is equal to I*(A*B)=(C*C)*I^T from matrix associativity.

You can validate this result by running the following numpy code:

import numpy as np
C = [[-1,0,1,0,0,0,0,0]]
for k in range(7):
    C.append(np.roll(C[0],k+1))
C = np.array(C)
#print(C)
I = np.transpose(np.array([[1,0,-1,0,1,0,1,1]]))

print(f'C=\n{C}\n')
print(f'C^2=\n{C@C}\n')

print(f'(C*C)*I=\n{(C@C)@I}\n')
print(f' C*(C*I)=\n{C@(C@I)}\n')

The result of running the above code is:

C=
[[-1  0  1  0  0  0  0  0]
 [ 0 -1  0  1  0  0  0  0]
 [ 0  0 -1  0  1  0  0  0]
 [ 0  0  0 -1  0  1  0  0]
 [ 0  0  0  0 -1  0  1  0]
 [ 0  0  0  0  0 -1  0  1]
 [ 1  0  0  0  0  0 -1  0]
 [ 0  1  0  0  0  0  0 -1]]

C^2=
[[ 1  0 -2  0  1  0  0  0]
 [ 0  1  0 -2  0  1  0  0]
 [ 0  0  1  0 -2  0  1  0]
 [ 0  0  0  1  0 -2  0  1]
 [ 1  0  0  0  1  0 -2  0]
 [ 0  1  0  0  0  1  0 -2]
 [-2  0  1  0  0  0  1  0]
 [ 0 -2  0  1  0  0  0  1]]

(C*C)*I=
[[ 4]
 [ 0]
 [-2]
 [ 1]
 [ 0]
 [-2]
 [-2]
 [ 1]]

 C*(C*I)=
[[ 4]
 [ 0]
 [-2]
 [ 1]
 [ 0]
 [-2]
 [-2]
 [ 1]]

So you can see that the same result is obtained.

Note that A*B is not [0 -2 0] but [1 0 -2 0 1] because you take A=[-1 0 1] (zero pad it on both sides to it has 2 zeros on each side) and move it as a sliding window over B=[-1 0 1] so first you get: [-1-1, 0-1 + -10, -11 + 00 + 1-1, 10 + 01, 11] = [1 0 -2 0 1]. The resulting convolution of 2 kernels of width n will be of length 2n-1.

simple 1D example showing convolution vs correlation to test commutative/associative property

1 Answers1