Convolutional layer in Python using Numpy - with Strides

Question

My question is highly related to this one, from which I copied the definition: Convolutional layer in Python using Numpy.

I am trying to implement a convolutional layer in Python using Numpy. The input is a 4-dimensional array of shape [N, H, W, C], where:

N: Batch size
H: Height of image
W: Width of image
C: Number of channels

The convolutional filter is also a 4-dimensional array of shape [F, F, Cin, Cout], where

F: Height and width of a square filter
Cin: Number of input channels (Cin = C)
Cout: Number of output channels

With no padding and a stride of S, the output should be a 4-dimensional array of shape [N, (H - F + 1) // S, (W - F + 1) // S, Cout].

For a fixed stride of S=1, this can be done efficiently with the following code:

from numpy.lib.stride_tricks import as_strided

def conv2d(a, b):
    Hout = a.shape[1] - b.shape[0] + 1
    Wout = a.shape[2] - b.shape[1] + 1

    a = as_strided(a, (a.shape[0], Hout, Wout, b.shape[0], b.shape[1], a.shape[3]), a.strides[:3] + a.strides[1:])

    return np.tensordot(a, b, axes=3)

Can anyone help me on how to implement a variable stride of S? As mentioned above I understand the output shape, just not the strides.

Yes, exactly! I want to make it work for any valid value of S — EliteKaffee, Dec 26 '20 at 23:13
To answer this I'd make make a smallish `a`, and try some alternatjve shape and strides in `as_strided`. While I have some experience with this, it's sufficiently distant that I have to combine reasoning with trial and error. — hpaulj, Dec 27 '20 at 01:30
Any chance you could give it a shot? I'm getting the shapes to work, but not the values — EliteKaffee, Dec 27 '20 at 16:16

score 2 · Answer 1 · answered Dec 29 '20 at 22:34

You can use the below implementation. Argument s represents the stride value.

from numpy.lib.stride_tricks import as_strided

def conv2d(a, b, s=1):
    Hout = (a.shape[1] - b.shape[0]) // s + 1
    Wout = (a.shape[2] - b.shape[1]) // s + 1
    Stride = (a.strides[0], a.strides[1] * s, a.strides[2] * s, a.strides[1], a.strides[2], a.strides[3])

    a = as_strided(a, (a.shape[0], Hout, Wout, b.shape[0], b.shape[1], a.shape[3]), Stride)

    return np.tensordot(a, b, axes=3)

# test
conv2d(x, kernel, 2)

Convolutional layer in Python using Numpy - with Strides

1 Answers1