Stack Overflow
Stack Overflow

Questions tagged [regularized]

Regularization involves introducing additional information in order to solve an ill-posed problem or to prevent over-fitting by shrinking the parameters of the model stay close to zero

In mathematics and statistics, particularly in the fields of machine learning and inverse problems, regularization involves introducing additional information in order to solve an ill-posed problem or to prevent overfitting. This information is usually of the form of a penalty for complexity, such as restrictions for smoothness or bounds on the vector space norm.

From http://en.wikipedia.org/wiki/Regularization_%28mathematics%29

<

195 questions
2
votes
1 answer

Python regularized gradient descent for logistic regression

I'm trying to implement Gradient Descent (GD) (not stochastic one) for logistic regression in Python 3x. And have some troubles. Logistic regression is defined as follows (1): logistic regression formula Formulas for gradients are defined as follows…
Mike
  • 21
  • 1
  • 3
2
votes
0 answers

Create a custom regularizer on weights of 2 layers

Here is small snippet of my code describing my custom regularizer that I want to implement. # Code adapted from https://github.com/keras-team/keras/issues/5563 class CustomRegularization(Layer): def __init__(self, **kwargs): …
Skand Vishwanath Peri
  • 173
  • 2
  • 10
2
votes
2 answers

TensorFlow: How to change keep_prob for Dropout without using a feed_dict

I have build a TensorFlow model that works with training and test batches provided by Input Queues. Thus, I am not explicitly feeding data for training using the standard feed_dict. Nevertheless, I need to implement dropout which requires a…
Squall
  • 31
  • 3
2
votes
0 answers

Efficient Way to Choose The Regularization Parameter

In the regularization problem, for example, LASSO, the regularization parameter, \lambda, is determined by cross validation (CV). When writing the code in R, I usually generate a sequence of values of \lambda. Then, for each one, I conduct the CV.…
vtshen
  • 202
  • 1
  • 11
2
votes
0 answers

How to code a regularized(penalized) multivariate logistic regression in R by using optimization function

I coded a regularized(penalized) multivariate logistic regression in R by using hand-written gradient function. It works fine, but the problem is that it is too slow. Is there any way to improve the speed by using an optimization function? I looked…
Ray Gold
  • 168
  • 1
  • 9
2
votes
1 answer

L2 Regularization must be added into cost function when using Linear Regression?

L2 Regularization must be added into cost function when using Linear Regression? Im not adding l2 or taking into account when computing cost. Is that wrong? The code snippet below should be sufficient : def gradient(self, X, Y, alpha,…
KenobiBastila
  • 539
  • 4
  • 16
  • 52
2
votes
1 answer

What is target in Python's sklearn coef_ output?

When I do ridge regression using sklearn in Python, the coef_ output gives me a 2D array. According to the documentation it is (n_targets, n_features). I understand that features are my coefficients. However, I am not sure what targets are. What is…
django_noob
  • 131
  • 1
  • 1
  • 5
2
votes
1 answer

Is it reasonable for l1/l2 regularization to cause all feature weights to be zero in vowpal wabbit?

I got a weird result from vw, which uses online learning scheme for logistic regression. And when I add --l1 or --l2 regularization then I got all predictions at 0.5 (that means all features are 0) Here's my command: vw -d training_data.txt…
zihaolucky
  • 196
  • 1
  • 11
1
vote
0 answers

Applying regularization in a custom layer in Keras/Tensorflow

I am trying to apply L2 regularization on the weights in a custom layer. class Linear(keras.layers.Layer): def __init__(self, units=32, input_dim=32, regularizer_amount=0.01, name="linear", **kwargs): super(Linear,…
uiag
  • 11
  • 2
1
vote
0 answers

Convergence warning: how to do action after it?

I am esing the ElasticNet from sklearn. With the typical commands enet = ElasticNet(alpha=a, l1_ratio=l, random_state=42, tol=1e-8) enet.fit(X_train, y_train) sometimes the model does not converge, i.e. I get the following ConvergenceWarning:…
user9875321__
  • 195
  • 12
1
vote
2 answers

Is passing activity_regularizer as argument to Conv2D() the same as passing it seperately right after Conv2D()? (Tensorflow)

I was wondering whether creating the model by passing activity_regularizer='l1_l2' as an argument to Conv2D() model = keras.Sequential() model.add(Conv2D(filters=16, kernel_size=(6, 6), strides=3, padding='valid', activation='relu', …
lonyen11
  • 91
  • 11
1
vote
0 answers

Why do we need regularization for linear least sqaures given that a line is the simplest model possible?

In linear least squares we are trying to fit a line to data. A line is the simplest model that can be fit to the data. How is it possible for a linear model to over-fit the data? In short why do we need regularization for fitting a line to a data?
user27665
  • 673
  • 7
  • 27
1
vote
1 answer

Changing regularization factor during training in Tensorflow

I wonder, is there an easy way? For example, changing learning rate can be easily done using tf.keras.optimizers.schedules: lr_schedule = tf.keras.optimizers.schedules.ExponentialDecay(0.001) optimizer =…
Legotin
  • 2,378
  • 1
  • 18
  • 28
1
vote
0 answers

Customized regularization function with customized training in tf.keras (TF2) using GradientTape

I want to define my own customised regulizer and I am making use of GradientTape. I am making use of the following code, however no matter how large I choose the tuning parameters to be, the results always stay the same. Does someone know how I can…
user11453289
  • 67
  • 6
1
vote
1 answer

Regularization function using weights from multiple layers?

I don't know if it is feasible but I'm asking just in case. Here is the (simplified) architecture of my model. Layer (type) Output Shape Param #Connected to ========================================== input_1 (InputLayer) [(None,…
Romain Xu-Darme
  • 13
  • 4
1 2 3
…
12 13