4

I understand regularization normally adds k*w^2 to the loss to penalize large weights. But in Keras there are two regularizer parameters - weight_regularizer and activity_ regularizer. What is the difference?

simon
  • 2,561
  • 16
  • 26
  • I don't find weight regularizer in specifications, but there is kernel regularizer. I assume that it is same since kernels are some kind of weights. Am I right? – Primoz Jun 14 '17 at 20:06

1 Answers1

6

The difference is that activity_regularizer is applied to the output from an intermediate layer, it penalizes large layer output.

Sergii Gryshkevych
  • 4,029
  • 1
  • 25
  • 42