2

I am new to AdaBoost algorithm. In sklearn SAMME algorithm's _boost_discrete() returns classifiers weight as "estimator_weight"

def _boost_discrete(self, iboost, X, y, sample_weight):
    .......

    return sample_weight, estimator_weight, estimator_error

But, for SAMME.R algorithm, "_boost_real()" is returning '1' instead of returning the estimator weight.

def _boost_real(self, iboost, X, y, sample_weight):
    .......

    return sample_weight, 1., estimator_error

My question is why SAMME.R algorithm returning the estimator_weight as '1'. I am following ref [1]. Please help me understanding the algorithm. Thanks in advance.

Ref: [1]J. Zhu, H. Zou, S. Rosset, T. Hastie, "Multi-class AdaBoost", 2009.

Community
  • 1
  • 1
Jaeger
  • 159
  • 4
  • 14

1 Answers1

12

Here's my understanding of why the weight may be set to '1' in the SAMME.R algorithm following this Multi-class Adaboost paper from 2006. The 2009 paper you mentioned does not include the SAMME.R algorithm.

In the paper (2006) sample_weight is represented by w and estimator_weight is represented by alpha.

If you take a look at Algorithm 2 SAMME

SAMME Algorithm

once the w's and alpha's are learned new samples are classified according to C(x). Note that the estimator_weight (alpha^(m)) appears in C(x) and can be interpreted as the strength of the update for weak learner m.

Now let's look at Algorithm 4 SAMME.R.

enter image description here

Note that estimator_weight (alpha) does not appear anywhere in this algorithm. Instead, the weak learners are characterized by the weighted class probabilities estimates and new samples are classified according to this new C(x). Even though estimator_weight (alpha) does not directly appear in C(x), one could place it in front of h_k^(m)(x) (as a multiplier) and define it to be 1 for all alpha:

enter image description here

This results in all the estimator_weight to take on a value of 1 at the end of training.

joceratops
  • 407
  • 5
  • 12