1

I'm able to understand how to code a binary SVM, for example a simple 1, -1 label. However I am going outside my comfort zone to try and perform multi-class and in effect multi-label SVM. However, I can't find anywhere how to do it.

I am going to use the iris data set, which has three classes. So how do you perform three lables? Is it simply -2, -1 , 1 or 2, 1 , -1?

For example, what differs in the way we train a SVM with two classes then having three. I am trying to implement this from scratch to really get a good understanding rather then just use libraries to get me through.

If anyone can find a good example, or possibly explain the process that would be fantastic. Thanks for your time

Willdomybest18
  • 127
  • 3
  • 12
  • 1
    I recommended looking into the One vs Rest and One vs One approach to multi-class classification. Python has a library called sklearn that has a lot of solid resources and information about such topics, along with the tools to implement (though it sounds like you'll abstain from using the latter). – gallen Jul 09 '20 at 01:40
  • 1
    You may find useful info in "sklearn multiclass svm function" https://stackoverflow.com/questions/49848453/sklearn-multiclass-svm-function – Roohollah Etemadi Jul 09 '20 at 01:49
  • @ thanks guys will have a look – Willdomybest18 Jul 09 '20 at 01:52

1 Answers1

1

I believe the sklearn library would be helpful here. The sklearn.svm.SVC class is "capable of performing binary and multi-class classification on a dataset.", according to the scikit-learn documentation (https://scikit-learn.org/stable/modules/svm.html).

The labels can actually take any set of integers, as long as they are distinct (e.g. {-1, 1, 2} and {0, 1, 2} and {1, 2, 3} are all valid). Generally, I believe it is best practice to use {0, 1, 2, ..., N} for your label assignments.

Please see the code below for an example:

import numpy as np
import matplotlib.pyplot as plt
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC

N = 1000  # Number of samples

# Create synthetic dataset
X1 = np.random.normal(loc=0, scale=1, size=(N, 2))
Y1 = 0 * np.ones(shape=(1000,))  # LABEL = 0 

X2 = np.random.normal(loc=[-5, 5], scale=1, size=(N, 2))
Y2 = 1 * np.ones(shape=(1000,))  # LABEL = 1

X3 = np.random.normal(loc=[5, -5], scale=1, size=(N, 2))
Y3 = 2 * np.ones(shape=(1000,))  # LABEL = 2

# Create stacked dataset
X = np.vstack((X1, X2, X3))
Y = np.hstack((Y1, Y2, Y3))

# TRAIN SVM LEARNING ALGORITHM
clf = SVC(kernel='linear')
clf = clf.fit(X, Y)

# create decision boundary plot
xx, yy = np.meshgrid(
    np.arange(-10, 10, 0.2),
    np.arange(-10, 10, 0.2))
Z = clf.predict(np.c_[xx.ravel(), yy.ravel()])
Z = Z.reshape(xx.shape)

# PLOT EVERYTHING
plt.scatter(X1[:,0], X1[:,1], color='r')
plt.scatter(X2[:,0], X2[:,1], color='b')
plt.scatter(X3[:,0], X3[:,1], color='y')
plt.contourf(xx,yy,Z,cmap=plt.cm.coolwarm, alpha=0.8)
plt.title("SVM With Linear Kernel and Three Labels (0, 1, 2)")
plt.show()

enter image description here Hope this helps!

Ryan Sander
  • 419
  • 3
  • 6
  • Thanks, although I want three labels. Here you have shown two labels which is easy enough to understand, but with three I don't know how the code changes. – Willdomybest18 Jul 09 '20 at 04:46
  • @Willdomybest18 my apologies for not clarifying. I've edited my answer above to reflect this, but in short I believe any label assignment works for multi-class SVM, as long as the label assignments are unique (e.g. {-1, 1, 2}, {0, 1, 2}, and {1, 2, 3} would all be appropriate for three-label SVM). – Ryan Sander Jul 09 '20 at 16:06