Should I use reguaization with Loss function or NN layer?

Question

I'm confused regarding the place of using regularization. In the theory, I saw regularization has been used with the Loss function.

But in the time implementation in Keras, I saw regularization has been used in the neural network layer.

from keras import regularizers
model.add(Dense(64, input_dim=64, kernel_regularizer=regularizers.l2(0.01)
model.add(Dense(28, input_dim=64, kernel_regularizer=regularizers.l1(0.05)

Here I used L1 and L2 loss in different layers. So How the final loss function will be calculated?

score 0 · Answer 1 · answered Jun 02 '22 at 18:05

Taken from Keras documentation:

Regularizers allow you to apply penalties on layer parameters or layer activity during optimization. These penalties are summed into the loss function that the network optimizes.

Indeed, it takes the error terms of l1/l2 regularization and adds them to the loss for that layer during the backpropagation.

Should I use reguaization with Loss function or NN layer?

1 Answers1