I am running various Regressions in Python with lots of variables. For a more sparse variable selection, I implemented a relaxed Lasso (https://relaxedlasso.readthedocs.io/en/latest/content.html#implementation).
The code works all fine and I get a more sparse variable selection and reasonable R-2 scores. The code is displayed below:
relaxed_lasso = RelaxedLassoLars(alpha=#result_from_RelaxedLassoLarsCV,theta=#result_from_RelaxedLassoLarsCV).fit(X_train, y_train.values.ravel())
print("Training set score: {:.2f}".format(relaxed_lasso.score(X_train, y_train.values.ravel())))
print("Test set score: {:.2f}".format(relaxed_lasso.score(X_test, y_test)))
print("Number of features used: {}".format(np.sum(relaxed_lasso.coef_ != 0)))
But I always get the following warning:
ConvergenceWarning: Regressors in active set degenerate. Dropping a regressor, after 10 iterations, i.e. alpha=1.165e-05, with an active set of 10 regressors, and the smallest cholesky pivot element being 1.825e-08. Reduce max_iter or increase eps parameters.
I would like to know what this means for the validity of my result. I couldn't find any meaninful explanation online. That's why I'm asking here.
Thanks!
