4

I am a non technical person, who is trying to implement image classification. In this paper, I came across the ADA Boost algorithm, which was implemented after the 'bag of features' step for video keyframes. Can someone explain in layman terms what the ADA Boost does, and what is its input and output? Can someone point me out to code for the same?

Pen Watson
  • 43
  • 1
  • 4

2 Answers2

4

First off, it would be nice to if you could link/name the paper you are referring to.

AdaBoost is a meta classification algorithm, as it combines multiple classifiers called weak learners. These weak learners are often really simple, e.g. they only classify the data based on one feature, and perform slightly better than random.

In image classification, AdaBoost will use as input a data set of images (with corresponding labels depicting to which class each sample belongs) and a set of weak learners. AdaBoost will then find the weak learner with the lowest error rate (i.e. best results) on the data. All correctly classified data samples are now given a lower weight as they are now less important, while the miss-classified samples are given a higher weight. AdaBoost will now start a new round and selects the best weak learner based on the newly weighted data. In other words, it will find a new weak learner which is better at classifying the samples which the previously selected weak learners were not able to classify.

The algorithm will continue with selecting these weak learners for a specified amount of iterations. The output consists of the group of selected weak learners. The learned classifier can now classify new images based on a majority vote of each weak classifier in the group (often the weak classifiers themselves are also weighted based on their achieved error rate).

You might want to take a look at software that have already implemented AdaBoost, like WEKA or the the computer vision orientated OpenCV.

Sicco
  • 6,167
  • 5
  • 45
  • 61
  • And what is the output of the classifier? For instance, Neural Network that utilize sigmoid function and trained to recognizes 2 classes (0 and 1) outputs the value between 0 and 1 that shows relative probability that detected context belongs to one of these classes. But what would be output from Adaboost algorithm for the same example? – supermus Jun 06 '13 at 14:59
  • 1
    @supermus AdaBoost can be used for classification or regression. The output in the classification case is one class from a fixed set of classes. In the regression case AdaBoost can output any number. – Sicco Jun 06 '13 at 16:09
1

Adaboost takes a bunch of weak classifiers and combines them to form a strong classifier. The outputs are a sequence of weights w_i for the weak classifiers used in the summand to form a single weighted classifier. There are many intermediate outputs from the algorithm, but maybe the most important is the weights themselves.

Although it wasn't originally conceived that way, Adaboost is equivalent to fitting a "forward stagewise" model on the training set, using the weak classifiers at each step using an exponential loss function: L(y,f(x)) = exp(-y*f(x)), where f(.) is our classifier. Viewed this way, some aspects of the algorithm are clearer. The exponential loss function is often used for classification problems, for good reason.

Charles Pehlivanian
  • 2,083
  • 17
  • 25