2

I'm implementing a custom regularizer myReg as a subclass of keras.regularizers.Regularizer. However, the computation of the regularization term needs the access to the original model that is being regularized. I checked the documentation of Keras and I didn't see there are things like self.model for me to access to the original model in the regularizer class. Since the regularizer needs to be created before the model is created, I can't even let the model to be an attribute of myReg. Is there any way to do it so that myReg can have an attribute like self.model? 

class myReg(Regularizer):
    def __init__(self, lambda):
        self.Lambda = lambda
        self.model = ?

    def __call__(self, x):
        regularization = 0
        y_pred = self.model.predict(x_train)
        regularization += self.Lambda * K.sum(x * y_pred) // just an example of using self.model in the regularizer class
        return regularization


model = Sequential()
model.add(Dense(128, activation='relu', input_dim=784, kernel_regularizer=myReg(0.01)))
model.add(Dense(10, activation='softmax'))

model.compile('adam', 'sparse_categorical_crossentropy', metrics=['accuracy'])
model.fit(x_train, y_train, 100, 10)

Thanks in advance! Any suggestion is appreciated!

Piggy Wenzhou
  • 305
  • 2
  • 10
  • Please provide some code. https://stackoverflow.com/help/minimal-reproducible-example – taha Jun 13 '20 at 02:45
  • @taha code added! – Piggy Wenzhou Jun 13 '20 at 03:02
  • Why are you trying to couple the regularizer with the model? This is not a good programming practice. Your regularizer must be standalone and process tensors independent of the model. Have you read custom regularizer part of this: https://keras.io/api/layers/regularizers/ – taha Jun 13 '20 at 03:08
  • @taha I'm trying to implement a custom regularizer too. I checked the official documents and found that all the regularizers are specified to act on a single layer. What if I wanna use a more complicated one that involves calculation between parameters in different layers? – PokeLu Jun 19 '20 at 08:33

0 Answers0