1

I am using l2_regularization Tensorflow has - tf.nn.l2_loss Can I use this?

  1. K.sum(K.square(K.abs(Weights)))
  2. tf.nn.l2_loss

Can I use this interchangeably in Keras (Tensorflow backend)?

desertnaut
  • 57,590
  • 26
  • 140
  • 166
Ayan Kumar Bhunia
  • 25
  • 4

1 Answers1

4

Yes, you can, but keep in mind that tf.nn.l2_loss computes output = sum(t ** 2) / 2 (from documentation), so you've forgotten about multiplying by 0.5. Also you don't have to calculate K.abs(weights) because K.square(K.abs(weights)) == K.square(weights).

The differences are:

  • tf.nn.l2_loss is implemented directly in kernel.

  • operations in Keras backend translate directly to Tensorflow defined here.

Andrzej Pisarek
  • 271
  • 3
  • 9