select best parameters for regression model using Gridsearch

Question

I'm doing linearregression modeling and i used gridsearch for select best parameters. below python steps i followed for this work but i got error (ValueError: Invalid parameter alpha for estimator LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None, normalize=False). Check the list of available parameters with estimator.get_params().keys().) please help for me to select best parameters for my model..

from sklearn.linear_model import LinearRegression
reg = LinearRegression()
parameters = {"alpha": [1, 10, 100, 290, 500],
              "fit_intercept": [True, False],
              "solver": ['svd', 'cholesky', 'lsqr', 'sparse_cg', 'sag', 'saga'], 
             }
grid = GridSearchCV(estimator=reg, param_grid = parameters, cv = 2, n_jobs=-1)
grid.fit(x_train, y_train)
reg.score(x_test,y_test)

There is no `alpha` parameter in sklearn's `LinearRegression()` — Celius Stingher, Dec 26 '19 at 14:38
thanks for your help..i'm beginner for machine learning. i'll correct that mistake. — randunu galhena, Dec 26 '19 at 14:42
@CeliusStingher Then how to tune best hyper param in case of LinearRegression? — Amit Modi, Jul 06 '20 at 18:54

Celius Stingher · Answer 1 · 2020-07-08T11:23:25.893

This is because there is no alpha parameter to be adjusted in sklearn.LinearRegression() therefore you are passing an invalid argument to a function, generating the ValueError you get.

You should look into this functions documentation to understand it better:

sklearn.linear_model.LinearRegression(*, fit_intercept=True, normalize=False, copy_X=True, n_jobs=None)

From here, we can see that hyperparameters we can adjust are fit_intercept, normalize, and n_jobs. Each function has its own parameters that can be tuned. Take for instance ExtraTreeRegressor (from extremely randomized tree regression model) also from sklearn:

sklearn.ensemble.ExtraTreesRegressor(n_estimators=100, *, criterion='mse', max_depth=None, min_samples_split=2, min_samples_leaf=1, min_weight_fraction_leaf=0.0, max_features='auto', max_leaf_nodes=None, min_impurity_decrease=0.0, min_impurity_split=None, bootstrap=False, oob_score=False, n_jobs=None, random_state=None, verbose=0, warm_start=False, ccp_alpha=0.0, max_samples=None)

As you can see the parameters that can be fine-tuned are a lot more, perhaps allowing for more flexibility in the model. Make sure you correctly understand each model and its tune-able parameters before applying GridSearchCV randomly.

select best parameters for regression model using Gridsearch

1 Answers1