5

I'm looking for a SVM implementation with support for non-linear kernels and one-vs-rest scenario, to perform a multi-label classification. Preferably, written in Python, or that I can call from Python with wrappers.

I was looking into sklearn, and there are two implementations to use SVM for classification:

sklearn.svm.LinearSVC - supports multi-label classification with a one-vs.-rest scenario, but it's based on liblinear, and therefore only supports linear kernels.

sklearn.svm.SVC - based on libsvm, supports non-linear kernels, but multi-label classification is done under a one-vs.-one reduction, it trains K (K − 1) / 2 binary classifiers for a K-way multiclass problem.

More info also here: http://scikit-learn.org/stable/modules/multiclass.html

Does anyone knows any other SVM implementations directly supporting multi-label classification and non-linear kernels ?

One possible solution could also be to adapt the code based on sklearn.svm.SVC, to perform One-vs-Rest, was this already attempted before?

David Batista
  • 3,029
  • 2
  • 23
  • 42
  • You mean multi-class, right? multi-label is something else (each sample is labeled with more than one label) – Itamar Katz Mar 21 '17 at 14:14
  • no, I mean actually multi-label; exactly that "each sample is labeled with more than one label" – David Batista Mar 21 '17 at 14:50
  • 1
    So how do you intend to use one-vs-one or one-vs-all? these are multi-class (single label) methods, not multi-label – Itamar Katz Mar 21 '17 at 20:06
  • OneVsRestClassifier: fits one classifier per class, the class is fitted against all the other classe. At prediction time, you apply each classifier (trained for each class) to decide if a given sample belongs to a class or not. OneVsOneClassifier: fits one classifier per pair of classes, it requires K * (k - 1) / 2 classifiers, it's much slower than OneVsRestClassifier. At prediction time the class which received the most votes is selected, or every class that is classified with a probability higher than a given threshold. – David Batista Mar 21 '17 at 20:58
  • I had an error on the formulation of my question, in the last sentence, just corrected it. – David Batista Mar 21 '17 at 21:07
  • I don't understand how you plant to convert OneVsRest and OneVsOne which are multi class methods into multi-label methods (and I don't think you do either) – carlosdc Mar 21 '17 at 21:13
  • Wouldn't this be a way of performing multi-label classification: training K classifiers in a OneVsRest scenario, and predicting each class with each trained classifier ? – David Batista Mar 21 '17 at 21:25
  • Maybe it's worth mentioning, I'm seeing this from the perspective of classifying a document (i.e. text) into different labels/topics. – David Batista Mar 21 '17 at 21:30
  • 1
    @carlosdc from [scikit-learn doc.](http://scikit-learn.org/stable/modules/generated/sklearn.multiclass.OneVsRestClassifier.html#sklearn.multiclass.OneVsRestClassifier) on OneVsRestClassifier: "This strategy can also be used for multilabel learning, where a classifier is used to predict multiple labels for instance, by fitting on a 2-d matrix in which cell [i, j] is 1 if sample i has label j and 0 otherwise. In the multilabel learning literature, OvR is also known as the binary relevance method...", however I am not sure of OneVsOneClassifier and its usage with multilabel classification. – PeterB Mar 23 '17 at 00:26

1 Answers1

3

Binary Relevance problem transformation method uses one-vs-rest approach for doing multi-label classification. One could easily implement SVM with non-linear kernels using scikit-multilearn library. The following is the sample python code to do the same, where each row of train_y is a one-hot vector representing multiple labels (For instance, [0,0,1,0,1,0])

from skmultilearn.problem_transform.br import BinaryRelevance
from sklearn.svm import SVC

# Non-linear kernel
svm = SVC(kernel='rbf')
cls = BinaryRelevance(classifier=svm)

cls.fit(train_x, train_y)
predictions = cls.predict(test_x)
Nikhil
  • 1,054
  • 9
  • 9