Run all regressors against the data in scikit

Question

I am working on creating a framework where I can call all regressors available in scikit-learn. Relating to this I have two questions-

How to get list of all regressors programmatically?
Objective is to run regressors against the dataset and acquire the metrics such as RMSE, R-Sq, Adjusted R-Sq, etc for model comparison, then apply hyper-parameter tuning and re-run again.

I am trying to replicate this functionality in Python-

https://github.com/tobigithub/caret-machine-learning/blob/master/caret-regression/caret-all-regression-models.R

I am sure this can be done in scikit. Would appreciate for any starting point.

Thanking in advance.

Just use RegressorMixin instead of ClassifierMixin in my answer here [list of all classification algorithms](https://stackoverflow.com/questions/41844311/list-of-all-classification-algorithms) — Vivek Kumar, Apr 02 '18 at 04:32

score 3 · Answer 1 · answered May 01 '20 at 16:01

I couldn't find a programmatic way to list all the regressors in sklearn, I have imported all of them then looped through them

from sklearn.ensemble.forest import RandomForestRegressor
from sklearn.ensemble.forest import ExtraTreesRegressor
from sklearn.ensemble.bagging import BaggingRegressor
from sklearn.ensemble.gradient_boosting import GradientBoostingRegressor
from sklearn.ensemble.weight_boosting import AdaBoostRegressor
from sklearn.gaussian_process.gpr import GaussianProcessRegressor
from  sklearn.isotonic import IsotonicRegression
from sklearn.linear_model.bayes import ARDRegression
from sklearn.linear_model.huber import HuberRegressor
from sklearn.linear_model.base import LinearRegression
from sklearn.linear_model.passive_aggressive import PassiveAggressiveRegressor 
from sklearn.linear_model.randomized_l1 import RandomizedLogisticRegression
from sklearn.linear_model.stochastic_gradient import SGDRegressor
from sklearn.linear_model.theil_sen import TheilSenRegressor
from sklearn.linear_model.ransac import RANSACRegressor
from sklearn.multioutput import MultiOutputRegressor
from sklearn.neighbors.regression import KNeighborsRegressor
from sklearn.neighbors.regression import RadiusNeighborsRegressor
from sklearn.neural_network.multilayer_perceptron import MLPRegressor
from sklearn.tree.tree import DecisionTreeRegressor
from sklearn.tree.tree import ExtraTreeRegressor
from sklearn.svm.classes import SVR
from sklearn.linear_model import BayesianRidge
from sklearn.cross_decomposition import CCA
from sklearn.linear_model import ElasticNet
from sklearn.linear_model import ElasticNetCV
from sklearn.kernel_ridge import KernelRidge
from sklearn.linear_model import Lars
from sklearn.linear_model import LarsCV
from sklearn.linear_model import Lasso
from sklearn.linear_model import LassoCV
from sklearn.linear_model import LassoLars
from sklearn.linear_model import LassoLarsIC
from sklearn.linear_model import LassoLarsCV
from sklearn.linear_model import MultiTaskElasticNet
from sklearn.linear_model import MultiTaskElasticNetCV
from sklearn.linear_model import MultiTaskLasso
from sklearn.linear_model import MultiTaskLassoCV
from sklearn.svm import NuSVR
from sklearn.linear_model import OrthogonalMatchingPursuit
from sklearn.linear_model import OrthogonalMatchingPursuitCV
from sklearn.cross_decomposition import PLSCanonical
from sklearn.cross_decomposition import PLSRegression
from sklearn.linear_model import Ridge
from sklearn.linear_model import RidgeCV
from sklearn.svm import LinearSVR

then you can loop through them accuracy = []

for i in range(len(Name)):
    regressor = globals()[Name[i]]

    Regressor = regressor(**param[i])
    Regressor.fit(X_train, y_train)
    y_pred = Regressor.predict(X_test)
    from sklearn.metrics import mean_squared_error
    import numpy as np
    Nans = np.isnan(y_pred)
    y_pred[Nans] = 0
    accuracy.append(np.sqrt(mean_squared_error(y_pred,y_test)))

you need to put the names of regressors into a list of names eg:

Name=[
'ExtraTreesRegressor',
'RandomForestRegressor']

Run all regressors against the data in scikit

1 Answers1