Can you get all estimators from an sklearn grid search (GridSearchCV)?

Question

I recently tested many hyperparameter combinations using sklearn.model_selection.GridSearchCV. I want to know if there is a way to call all previous estimators that were trained in the process.

search = GridSearchCV(estimator=my_estimator, param_grid=parameters)
# `my_estimator` is a gradient boosting classifier object
# `parameters` is a dictionary containing all the hyperparameters I want to try

I know I can call the best estimator with search.best_estimator_, but I would like to call all other estimators as well so I can test their individual performance.

The search took around 35 hours to complete, so I really hope I do not have to do this all over again.

NOTE: This was asked a few years ago (here), but sklearn has been updated multiple times since and the anwer may be different now (I hope).

Answer 1

No, none of the tested models are saved, except (optionally, but by default) one final one trained on the entire training set, your best_estimator_. Especially when models store significant amounts of data (e.g. KNNs), saving all the fitted estimators would be very memory-expensive, and usually not of much use. (cross_validate does have a parameter return_estimator, but the hyperparameter tuners do not. If you have a compelling reason to add it, it probably wouldn't take much work and you could open a GitHub Issue at sklearn.)

However, you do have the cv_results_ attribute that documents the scores of all of the tested estimators. That's usually enough for inspection purposes.