11

I want to perform multi-class classification using the svm function of e1071 package. But from what I came to know from the documentation of svm, it can only perform binary classification. The vignettes document tells this for multi-class classification: "To allow for multi-class classification, libsvm uses the one-against-one technique by fitting all binary subclassifiers and finding the correct class by a voting mechanism".
What I still don't understand is if we can perform the multi-class classification with svm of e1071 in R? If yes, please explain how we can do it over iris dataset.

StrikeR
  • 1,598
  • 5
  • 18
  • 35
  • 1
    The very first example in help("svm") does exactly what you're after. – fabians Feb 25 '14 at 10:56
  • @fabians yeah you are right about the example and I have seen it earlier. So, does it mean that the `svm` of `e1071` can perform multi-class classification and not limited to binary classification? Does this function work the same way on the dataset with 10 output classes as `iris` has 3? – StrikeR Feb 25 '14 at 11:16
  • Replied for similar question in another thread. Below is the link: http://stackoverflow.com/a/37697836/4861626 – S Pavan Kumar Jun 08 '16 at 09:02

2 Answers2

22

The iris dataset contains three class labels: "Iris setosa", "Iris virginica" and "Iris versicolor". To employ a balanced one-against-one classification strategy with svm, you could train three binary classifiers:

The first classifier's training set only contains the "Iris setosa" and "Iris virginica" instances. The second classifier's training set only contains the "Iris setosa" and the "Iris versicolor" instances. The third classifier's training set--I guess by now you'll know already--contains only the "Iris virginica" and the "Iris versicolor" instances.

To classify an unknown instance, you apply all three classifiers. A simple voting strategy could then select the most frequently assigned class label, a more sophisticated may also consider the svm confidence scores for each assigned class label.

Edit (This principle works out of the box with svm):

# install.packages( 'e1071' )
library( 'e1071' )
data( iris )
model <- svm( iris$Species~., iris )
res <- predict( model, newdata=iris )
idleherb
  • 1,072
  • 11
  • 23
  • 2
    that is a very nice way of approaching the problem. But, before going that far, I want to know if the `svm` of `e1071` can directly perform this multi-class classification. – StrikeR Feb 25 '14 at 11:18
  • the reason I'm asking is that, if I have a data with 10 output classes I need to model 10C2 (10 combination 2) = 45 such classifiers, which is a huge task. – StrikeR Feb 25 '14 at 11:28
  • 4
    Ok :) So the short answer is yes, `svm` can also do multiclass classification and it works the same way as for binary (see edit in my answer). – idleherb Feb 25 '14 at 11:49
1

R document says that "For multiclass-classification with k levels, k>2, libsvm uses the ‘one-against-one’-approach, in which k(k-1)/2 binary classifiers are trained; the appropriate class is found by a voting scheme."

Hima Bindu
  • 11
  • 1