2

Is there a sample python code for 2-class SVM classification using the custom kernel or sigmoid kernel?

The code below uses 3-class classification. How to modify this to a 2-class SVM?

http://scikit-learn.org/stable/auto_examples/svm/plot_custom_kernel.html

import numpy as np
import matplotlib.pyplot as plt
from sklearn import svm, datasets

iris = datasets.load_iris()
X = iris.data[:, :2]  # we only take the first two features. We could
                      # avoid this ugly slicing by using a two-dim dataset
Y = iris.target

def my_kernel(X, Y):
    """
    We create a custom kernel:

                 (2  0)
    k(X, Y) = X  (    ) Y.T
                 (0  1)
    """
    M = np.array([[2, 0], [0, 1.0]])
    return np.dot(np.dot(X, M), Y.T)


h = .02  # step size in the mesh

# we create an instance of SVM and fit out data.
clf = svm.SVC(kernel=my_kernel)
clf.fit(X, Y)

# Plot the decision boundary. For that, we will assign a color to each
# point in the mesh [x_min, x_max]x[y_min, y_max].
x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1
y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1
xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))
Z = clf.predict(np.c_[xx.ravel(), yy.ravel()])

# Put the result into a color plot
Z = Z.reshape(xx.shape)
plt.pcolormesh(xx, yy, Z, cmap=plt.cm.Paired)

# Plot also the training points
plt.scatter(X[:, 0], X[:, 1], c=Y, cmap=plt.cm.Paired, edgecolors='k')
plt.title('3-Class classification using Support Vector Machine with custom'
          ' kernel')
plt.axis('tight')
plt.show()
Tsyvarev
  • 60,011
  • 17
  • 110
  • 153
Christel Junco
  • 123
  • 1
  • 2
  • 15
  • Just use the data for first two classes. The iris dataset has first 50 samples for class1, next 50 for class2 and last 50 for class3. You can use only the first 100 samples. Do `X = X[:100]` and `Y = Y[:100]` – Vivek Kumar Feb 01 '18 at 05:03

1 Answers1

0

You have 3 classes, since your data is categorized into 3 classes. You can manipulate the target vector to only contain two classes.

Y = iris.target
for index, value in enumerate(Y):
  Y[index] = value % 2

But this is a bit of an ugly hack, since it effectively merges every class > 1 into class 1.

Finn
  • 1,999
  • 2
  • 24
  • 29
  • woow, I run it and it works! but can I ask or can you explain the code above you've answered, where the index and value come from? – Christel Junco Jan 31 '18 at 09:33
  • Sure. Y are the target labels (the classes) for your sample data. In your case they have values 0, 1 or 2. My code loops through Y (which you can do as it is a [ndarray](https://docs.scipy.org/doc/numpy-1.12.0/reference/generated/numpy.ndarray.html).) I use [enumerate](https://docs.python.org/2/library/functions.html#enumerate) to keep track of the index and update the value [modulo](https://en.wikipedia.org/wiki/Euclidean_division) 2 to end up with an array containing only 0 and 1. – Finn Jan 31 '18 at 12:30
  • ah okay, I understand, thank you for that. How about the variable 'M'? what is the purpose/function of that? if I remove it, the variable 'h' will be having an error and so on to the variable 'clf'. – Christel Junco Jan 31 '18 at 19:50
  • Hi, your kernel gets as Input matrices of the shape `(m, 2), (n, 2)` (Since this example is using two features.) And you want to get an output of shape `(m, n)`. `M` has shape (2, 2). Hence if you do `X * M * Y^T` you end up with the desired shape. You can do anything you want with the inputs (that's the key purpose of the kernel trick). As long as you return the right shape. So if you want to remove `M` you can just return `np.dot(X, Y.T)` here `M` is implicitly the identity matrix. – Finn Feb 03 '18 at 14:25
  • Also you might want to [watch this MIT OpenCourseWare video about SVMs](https://www.youtube.com/watch?v=_PwhiWxHK8o) to get the intuition. – Finn Feb 03 '18 at 14:29