Adaboost Implementation with Decision stump

Question

I have been trying to implement Adaboost using decision stump as weak classifier but i do not know how to give preference to the weighted miss classified instances?

Answer 1

A decision stump is basically a rule that specifies a feature, a threshold and a polarity. So given samples you have to find the one feature-threshold-polarity combination that has the lowest error. Usually you count the misclassifications and divide it by the number of samples to get the error. In Adaboost a weighted error used, which means that instead of counting the misclassifications, you sum up the weights that are assigned to the misclassified samples. I hope this is all clear so far.

Now, to give a higher preference to misclassified sample in the next round you adjust the weights assigned to the samples by either increasing the weights of the misclassified samples or decreasing the weights of the correctly classified ones. Assume that E is your weighted error, you multiply the misclassified sample weights by the value (1-E)/E. Since the decision stump is better than random guessing, E will be < 0.5 which means that (1-E)/E will be > 1, so that the weights are increased (e.g. E = 0.4 => (1-E)/E = 1.5). If on the other hand, you want to decrease the correctly classified sample weights, use E/(1-E) instead. However, do not forget to normalized the weights afterwards so that they sum up to 1. This is important for the computation of the weighted error.