Stack Overflow
Stack Overflow

Questions tagged [regularized]

Regularization involves introducing additional information in order to solve an ill-posed problem or to prevent over-fitting by shrinking the parameters of the model stay close to zero

In mathematics and statistics, particularly in the fields of machine learning and inverse problems, regularization involves introducing additional information in order to solve an ill-posed problem or to prevent overfitting. This information is usually of the form of a penalty for complexity, such as restrictions for smoothness or bounds on the vector space norm.

From http://en.wikipedia.org/wiki/Regularization_%28mathematics%29

<

195 questions
2
votes
2 answers

Use specific alpha parameters for multiple elastic net in R

I'm trying to perform multiple elastic net simultaneously in R. I have a 10x15 matrix each column is called Xi e.g. X1,X2,...,X15 and perform elastic net in order to obtain the optimal alpha and lambda parameters. Then i save the alpha values in a…
nickolakis
  • 621
  • 3
  • 7
2
votes
1 answer

Neural Network regularizer L1 and L2

I'm doing music genre classification. I built a file .h5 with my model which is a neural network. And now I want to use it. Here is the code to predict the music genre : #%% import librosa import tensorflow as tf import numpy as np from collections…
Hofman
  • 51
  • 1
2
votes
2 answers

How to take the trainable parameters into a loss function in Tensoflow.Keras

I'm trying to implement a loss function where variables in Convolutional layers are required for calculation. There's one method given by the official documents that involve variables in the loss function: If this is not the case for your loss…
PokeLu
  • 767
  • 8
  • 17
2
votes
0 answers

Keras Regularizer class: access to the original model

I'm implementing a custom regularizer myReg as a subclass of keras.regularizers.Regularizer. However, the computation of the regularization term needs the access to the original model that is being regularized. I checked the documentation of Keras…
Piggy Wenzhou
  • 305
  • 2
  • 10
2
votes
2 answers

What does sklearn.linear_model RidgeCV normalize= parameter exactly do

I am confused by what normalized= exactly do in RidgeCV from sklearn.linear_model. The documentation says: normalize : bool, default=False This parameter is ignored when fit_intercept is set to False. If True, the regressors X will…
Sarah
  • 1,854
  • 17
  • 18
2
votes
1 answer

Sklearn StandardScaler + ElasticNet with interpretable coefficients

In order to properly fit a regularized linear regression model like the Elastic Net, the independent variables have to be stanardized first. However, the coefficients have then different meaning. In order to extract the proper weights of such model,…
foxale
  • 127
  • 9
2
votes
1 answer

Shouldn't H2O standardize categorical predictors for regularized GLM models (lasso, ridge, elastic net)?

"The lasso method requires initial standardization of the regressors, so that the penalization scheme is fair to all regressors. For categorical regressors, one codes the regressor with dummy variables and then standardizes the dummy variables" (p.…
Elliot
  • 21
  • 3
2
votes
1 answer

Is there any regularizer that penalizes redundant neurons?

I trained a network on a real-value labels (floating point numbers from 0.0 to 1.0) - several residual blocks in the beginning, and the last layers are fully-connected layer with 64 neurons + ELU activation, fully-connected layer with 16 neurons +…
Emil
  • 629
  • 2
  • 7
  • 24
2
votes
1 answer

add_loss() to regularize layer activity/weights not working anymore with tensorflow.keras 2.0 update

I was previously adding regularization of activations and/or kernels in Tensorflow.keras on a pretrained network, using a loop over layers: if regul_what is 'kernel': for layer in model.layers: if isinstance(layer, DepthwiseConv2D): …
shora
  • 131
  • 11
2
votes
1 answer

output layer regularization implementation

I’m building a NN model using keras, and I wish to impose a constraint on it that doesn’t (directly) have to do with the weights. Would be very grateful for some help / points me towards some relevant keywords to look up. The constraint I wish to…
Mayoman
  • 25
  • 8
2
votes
1 answer

How to create an autoencoder where each layer of encoder should represent the same as a layer of the decoder

I want to build an autoencoder where each layer in the encoder has the same meaning as a correspondent layer in the decoder. So if the autoencoder is perfectly trained, the values of those layers should be roughly the same. So lets say the…
Felix Unverzagt
  • 23
  • 5
2
votes
2 answers

How to regularize a layer's kernel weights bias weights in a single regularization function?

The Keras documentation introduces separate classes for weight regularization and bias regularization. These can be subclasses to add a custom regularizer. An example from the Keras docs: def my_regularizer(x): return 1e-3 *…
YoavEtzioni
  • 85
  • 10
2
votes
1 answer

Why is theta0 skipped while performing regulariztion on regression?

I am currently learning ML on coursera with the help of course on ML by Andrew Ng. I am performing the assignments in python because I am more used to it rather than Matlab. I have recently come to a problem regarding my understanding of the topic…
Aditya
  • 61
  • 7
2
votes
2 answers

keras custom activity regularizer

def kl_divergence(p, p_hat): return (p * K.log(p / p_hat)) + ((1 - p) * K.log((1 - p) / (1 - p_hat))) class SparseActivityRegularizer(Regularizer): sparsityBeta = None def __init__(self, l1=0., l2=0., p=0.01, sparsityBeta=0.1): self.p…
Shivam Chandhok
  • 21
  • 1
  • 2
2
votes
1 answer

How to use regularization with Dynamic_rnn

I want to use l2-regularizatin with Dynamic_rnn in tensorflow but it seems this is not handled gracefully currently. While loop is the source of error. Below is a sample code snippet to reproduce the problem import numpy as np import tensorflow as…
figs_and_nuts
  • 4,870
  • 2
  • 31
  • 56
1 2 3
…
12 13