Get feature importance from GridSearchCV

Question

Is there a way to get feature importance from a sklearn's GridSearchCV?

For example :

from sklearn.model_selection import GridSearchCV
print("starting grid search ......")
optimized_GBM = GridSearchCV(LGBMRegressor(),
                             params,
                             cv=3,
                             n_jobs=-1)
# 
optimized_GBM.fit(tr, yvar)
preds2 = optimized_GBM.predict(te)

Is there a way I can access feature importance ?

Maybe something like

optimized_GBM.feature_importances_

score 34 · Answer 1 · answered Apr 13 '18 at 13:56

34

This one works

optimized_GBM.best_estimator_.feature_importances_

answered Apr 13 '18 at 13:56

Aptha Gowda

968
2
10
20

16

In addition: If you are using a pipeline, .i.e. your estimator is a pipeline object, you have to add the pipeline step name: `optimized_GBM.best_estimator_.named_steps["step_name"].feature_importances_` – 00schneider Sep 27 '19 at 11:48
3

@aptha-gowda is there a way to also extract the feature names? i.e. the name of the variable? – Jeremy K. Dec 15 '20 at 02:48
@00schneider if I do PCA then fit a model in my pipeline, how do I recover the importance of the original variables in the model. – BND Feb 26 '21 at 20:42

score 21 · Accepted Answer · edited Nov 27 '19 at 12:21

21

Got it. It goes something like this :

optimized_GBM.best_estimator_.feature_importance()

if you happen ran this through a Pipeline and receive object has no attribute 'feature_importance' try optimized_GBM.best_estimator_.named_steps["step_name"].feature_importances_

where step_name is the corresponding name in your pipeline

edited Nov 27 '19 at 12:21

Areza

5,623
7
48
79

answered Jan 22 '18 at 08:30

Nick M

822
1
7
20

4

Did you maybe got `object has no attribute 'feature_importance'` error? – limitless Jul 31 '18 at 10:48
1

No,I did not get any error. This worked for me. I was using python 3.6. However, this was in Jan so the function call might have changed as suggested by other answer. – Nick M Aug 01 '18 at 08:36

score 2 · Answer 3 · answered May 27 '20 at 07:23

2

That depends on what model you have selected. If you choose a SVM you wont be having feature importance parameter, but in decision trees you will get it

answered May 27 '20 at 07:23

Akash C

31
5

Get feature importance from GridSearchCV

3 Answers3