5

I am working on a fuzzy convolution filter for CNNs. I have the function ready - it takes in the 2D input matrix and the 2D kernel/weight matrix. The function outputs the convolved feature or the activation map.

Now, I want to use Keras to build the rest of the CNN that will have the standard 2D convolution filters too.

Is there any way I can insert my custom filter into the Keras model in such a way that the kernel matrix is updated by the built in libraries of the Keras backend? Alternatively is there any library that I can use to update the kernel with every iteration?

Rangan Das
  • 323
  • 2
  • 11

2 Answers2

13

Suppose we want to apply a 3x3 custom filter onto an 6x6 image.

enter image description here

Necessary import 

import keras.backend as K
import numpy as np
from keras import Input, layers
from keras.models import Model

Definition of the custom filter

enter image description here

# custom filter
def my_filter(shape, dtype=None):

    f = np.array([
            [[[1]], [[0]], [[-1]]],
            [[[1]], [[0]], [[-1]]],
            [[[1]], [[0]], [[-1]]]
        ])
    assert f.shape == shape
    return K.variable(f, dtype='float32')

Dummy example input image (it is 1 channel image. So dimension will be 6x6x1 . Here, pixel values are random integer. Generally pixel values should be 0 to 255 or 0.0 to 1.0.)

input_mat = np.array([
    [ [4], [9], [2], [5], [8], [3] ],
    [ [3], [6], [2], [4], [0], [3] ],
    [ [2], [4], [5], [4], [5], [2] ],
    [ [5], [6], [5], [4], [7], [8] ],
    [ [5], [7], [7], [9], [2], [1] ],
    [ [5], [8], [5], [3], [8], [4] ]
])

# we need to give the batch size. 
# here we will just add a dimension at the beginning which makes batch size=1
input_mat = input_mat.reshape((1, 6, 6, 1))

Dummy conv model where we will use our custom filter 

def build_model():
    input_tensor = Input(shape=(6,6,1))

    x = layers.Conv2D(filters=1, 
                      kernel_size = 3,
                      kernel_initializer=my_filter,
                      strides=2, 
                      padding='valid') (input_tensor)

    model = Model(inputs=input_tensor, outputs=x)
    return model

Testing 

model = build_model()
out = model.predict(input_mat)
print(out)

Output

[[[[ 0.]
   [-4.]]

  [[-5.]
   [ 3.]]]]
Uzzal Podder
  • 2,925
  • 23
  • 26
  • Thanks for this. However I'm wondering if this will remain "frozen" when training a network which contains this filter in one layer. For example, one convolutional layer has the fixed filters (so no weights to be learned), then the remaining layers require coefficients to be learned. Does this example work in this case? – Dorian Dec 08 '19 at 01:52
  • 3
    The custom filter will be **trainable** by default. But if you use `layers.Conv2D(..., kernel_initializer=my_filter, trainable=False)` then it will be **frozen**. So, you can make it trainable or non-trainable whatever you want. – Uzzal Podder Dec 16 '19 at 06:16
  • 1
    here is an example - https://colab.research.google.com/drive/12hLFZKAf-4p-mkVpzWz78dz5PHRPPgsz – Uzzal Podder Dec 16 '19 at 06:22
  • 1
    How can you define a wide variety of filters? So normally many filters and their combinations can be defined in conv2d, right? – acs Feb 16 '21 at 17:03
0

You could write your own layer class. You can find a good documentation about it here. Furthermore, you can start by using the code of the Conv2D class. Now, you only have to change the way how the kernel is initially created. This way, your filters will be updated according to the backpropagation. If you do not want your custom filter to change you must create a new variable (which only contains your filter), calculate the convolution using K.conv(...) and concatenate this result with the usual output.

zimmerrol
  • 4,872
  • 3
  • 22
  • 41
  • Can you give a small example of this ("If you do not want your custom filter to change you must create a new variable (which only contains your filter), calculate the convolution using K.conv(...) and concatenate this result with the usual output."), using sin and cos as a rotation filter, please? That would not require any updates and would help consolidate the idea. Thank you. – Moondra Mar 01 '19 at 17:21
  • @Moondra did you figure this out? – nzy Mar 28 '19 at 13:52
  • @enjal No, I am actually searching again for an answer. – Moondra May 11 '19 at 22:58