Can I create a custom regularization term (Keras) using not only the weight matrix as a parameter?

Question

As stated in the title, I would like to be able to penalize my model weights by creating a custom regularization term.

ex:

def customized_regularizer(weight_matrix, parameterA, parameterB):
    return(K.sum(K.dot(parameterA, weight_matrix) - parameterB))

model.add(Dense(64, input_dim=64,
                kernel_regularizer=customized_regularizer))

However, while looking at the Keras doc (https://keras.io/regularizers/), I see:

"Developing new regularizers Any function that takes in a weight matrix and returns a loss contribution tensor can be used as a regularizer, e.g..:"

Is it thus possible to create such a custom regularizer?

score 1 · Answer 1 · answered Feb 28 '20 at 09:12

Yes, you can do it this way:

def custom_reg_builder(parameterA, parameterB):

    def custom_reg(weight_matrix):
        return(K.sum(K.dot(parameterA, weight_matrix) - parameterB))

    return custom_reg


# ...

model.add(Dense(64, input_dim=64, kernel_regularizer=custom_reg_builder(0.1, 0.01)))

Can I create a custom regularization term (Keras) using not only the weight matrix as a parameter?

1 Answers1