0

I know you can AdaBoost with multiple instances of a single model (e.g., 600 Decision Trees, Bayesian Ridges, or Linear Models). Is it possible to AdaBoost with a gauntlet of models at the same time, and how?

AdaBoost([DecisionTree, BayesianRidge, LinearRegressor, ...])

Each standalone model has its pros and cons, and I was wondering if it was possible to mash them all together under an umbrella.

Clayton
  • 3
  • 3

1 Answers1

0

Since AdaBoost is decoupled from the weak classifier's implementation: Yes, you can. These architectures are called hybrid ensembles. However, it's not always needed as AdaBoost only requires the weak classifiers to be slightly better than random. Thus, it's often enough to have a homogenous ensemble of classifiers. In the end you'll have to test it for your own scenario if it gives you a performance boost.

runDOSrun
  • 10,359
  • 7
  • 47
  • 57
  • 1
    Can you provide a simple code implementation? I can't seem to find anything for doing this. – Clayton Feb 23 '15 at 21:07
  • In scikit? I'm not sure if it's possible since it's using a `base_parameter`. You might have to modify it, write it yourself or find a different library. – runDOSrun Feb 23 '15 at 21:18