Stack Overflow
Stack Overflow

Questions tagged [regularized]

Regularization involves introducing additional information in order to solve an ill-posed problem or to prevent over-fitting by shrinking the parameters of the model stay close to zero

In mathematics and statistics, particularly in the fields of machine learning and inverse problems, regularization involves introducing additional information in order to solve an ill-posed problem or to prevent overfitting. This information is usually of the form of a penalty for complexity, such as restrictions for smoothness or bounds on the vector space norm.

From http://en.wikipedia.org/wiki/Regularization_%28mathematics%29

<

195 questions
0
votes
1 answer

Why there is a difference between the accuracy of sklearn.LogisticRegression with penalty='l1' and 'l2' and C=1e80?

I am somewhat disappointed by the results I am getting. I create two models (sklearn.linear_models.LogisticRegression) with C=1e80 and penalty = 'l1' or 'l2', and then test them using sklearn.cross_validation.cross_val_score with cv=3 and…
Mikhail Akimov
  • 1
  • 1
0
votes
1 answer

R Remove intercepts in logistic regression

I am using the rms library to perform regularized logistic regression, and wish to force the intercept to zero. I'm using the following to simulate and regress: library(rms) N = 100 pred <- vapply(1:12, function(i) rnorm(N, mean = 0, sd =1),…
TreeStump
  • 63
  • 6
0
votes
1 answer

ridge regression: test error goes up then down as the training sample increases (from underdetermined to overdetermined)

I am looking into the effect of the training sample size when doing a ridge (regularised) regression. I get this very strange graph when I plot the test error versus the train set size: . The following code generates a training set and a test set…
asachet
  • 6,620
  • 2
  • 30
  • 74
0
votes
2 answers

asymmetric regularization in machine learning libraries (e.g. scikit ) in python

The problem requires me to regularize weights of selected features while training a linear classifier. I am using python SKlearn. Having googled a lot about incorporating asymmetric regularization for classifiers in SKlearn, I could not find any…
Deepali Semwal
  • 139
  • 1
  • 2
  • 9
0
votes
0 answers

Good MSE doesnt imply good prediction in logistic regression?

I am writing some code for regularized logistic regression. I observe this interesting phenomena and wonder if it is a normal thing or just my code is wrong. For loss function, I am using the logistic loss function ( maximize likelihood of binary…
Bluesking
  • 11
  • 1
0
votes
1 answer

Weird phenomenon with SVM: negative examples score higher

I use the VL-Feat and LIBLINEAR to handle the 2-category classification. The #(-)/#(+) for the training set is 35.01 and the dimension of each feature vector is 3.6e5. I have around 15000 examples. I have set the weight of positive example to be…
Peiyun
  • 171
  • 1
  • 2
  • 13
0
votes
1 answer

regularization path for SVM in python

I have looked around to find a R's svmpath equivalent for sklearn or python. Did I overlook it or do I need to go with R for this task. Thanks
user17375
  • 529
  • 4
  • 14
0
votes
1 answer

what should be the parametric form of the l2 regularization in a Bayesian setting?

In a Bayesian setting for parameter estimation, what should be the parametric form of the prior distribution in order to perform l2 regularization?
user966892
-1
votes
1 answer

How to implement regularization

My task was to implement model parameter tuning using stochastic gradient descent. Below is my function implementation code. However, I would like to add any regularization. def gradient(X, y, w, batch, alpha): gradients = [] error =…
villerpa
  • 1
-1
votes
1 answer

L2 regularization in Logistic regression vs NN

To implement L2 regularization for logistic regression we add L2 norm to the base loss: With multilayer Neural Networks we do the same, but additionally, we increase per loss weight derivative of the weight during backward propagation: The…
Bohdan Turani
  • 19
  • 7
-1
votes
1 answer

LASSO regression implementation without python libraries

I am a python newbie and seriously searching for a python implementation of LASSO without the use of python libraries (e.g sklearn etc.) I am particularly interested in this to help me understand how the underlying maths translates to python code.…
Ziggey
  • 13
  • 7
-1
votes
1 answer

Training accuracy vs test accuracy

I am training a CNN and I am getting results of 85% accuracy in the training set, and 65% accuracy in the test set. Is it okey to assume that, with a proper setting of the regularization of the network (dropout and L2 in my case), my test accuracy…
sdiabr
  • 487
  • 1
  • 8
  • 23
-1
votes
1 answer

Assigning location values based on date and time of a second data set

I have 2 data frames: 1 is a sequence of GPS locations with associated date-times (POSIXct) GPS <- data.frame(Lat=c(58.65209, 58.65183, 58.65142, 58.65129, 58.65126, 58.65124, 58.65122, 58.65119, 58.65117, 58.65115), …
Jojo
  • 4,951
  • 7
  • 23
  • 27
-3
votes
2 answers

What does (Theta1(:,2:end).^2,2) mean in this code?

What does (Theta2(:, 2:end).^2, 2) mean? p = sum(sum(Theta1(:, 2:end).^2, 2))+sum(sum(Theta2(:, 2:end).^2, 2));
Sudarshan
  • 1
  • 2
1 2 3
…
12
13