3

I am running below code to generate all the coefficient positive:

from sklearn.linear_model import Lasso

pos = Lasso(positive=True)
pos.fit(X,y)
list(pos.coef_)

the above code give me positive coefficient or "0" but I need all to be positive with some positive impact.

Requirement = All Positive coefficient(Coefficient should not be Zero(0))

How can I perform above task?

kevins_1
  • 1,268
  • 2
  • 9
  • 27
Ghanshyam Savaliya
  • 568
  • 5
  • 22
  • You could try removing the factors where the regression coefficient is zero and then re-running the regression again (feature elimination). The value of zero has the effect of implicitly removing the feature from the regression, so this would have the same effect. – James Phillips Sep 13 '19 at 08:37
  • 1
    @JamesPhillips, Thanks for this but Is it possible for you to provide some example through some basic code? – Ghanshyam Savaliya Sep 13 '19 at 09:58
  • As an example: we regress factors X1, X2 and X3 against Y as in the equation "Y = aX1 + bX2 + cX3". In this example the regression gives us values for the parameters a, b, and c with parameter c equal to 0.0. This means that no matter what we use for X3, the value of "c * X3" is zero every time and has no effect on our calculations. We can simply remove X3 from the regression and have the same result as leaving it in. – James Phillips Sep 13 '19 at 11:22

2 Answers2

1
lasso = Lasso(alpha=1, positive=True)
lasso_coeff['Coefficient Estimates] = pd.Series(lasso.coef_)
print(lasso_coeff)

# The above set of lines will force the coefficients to positive.
VirtualScooter
  • 1,792
  • 3
  • 18
  • 28
Indu
  • 11
  • 1
0

Lasso does not solve the l0-penalized least squares but instead l1-penalized least squares. The solution you get for alpha=0.01 is the Lasso solution (with a single non zero coef of ~0.245 for feature #10).

Even if your solution has a squared reconstruction error of 0.0, it still has a penalty of 1.0 (multiplied by alpha).

The solution for lasso with alpha=1.0 has a small squared reconstruction error of 0.04387 (divided by 2 * n_samples == 6) and a smaller l1 penalty of 0.245 (multiplied by alpha).

The objective function minimized by lasso is given in the docstring:

http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Lasso.html

To summarize the different priors (or penalties) commonly used to regularize least squares regression:

  • l2 penalty favors any number of non-zero coefficients but with very small absolute values (close to zero)

  • l1 penalty favors a small number of non-zero coefficients with
    small absolute values.

  • l0 favors a small number of non zero coefficients of any absolute
    value.

l0 being non-convex, it is often not as easy to optimize as l1 and l2. This is why people use l1 (lasso) or l1 + l2 (elastic net) in practice to find sparse solutions even if not as clean as l0.

Happy Patel
  • 2,259
  • 1
  • 16
  • 25