I am looking into support vector machines and I am wondering what the difference between the decision boundary and the optimal hyperplane is? They both seem to be described as the line drawn to separate the datapoints.
Asked
Active
Viewed 8,220 times
8
-
3How is this a Python question? – martineau Dec 18 '13 at 22:00
-
2I was using scikit-learn, a python module – High schooler Dec 18 '13 at 23:40
-
What part of your question was specific to scikit-learn and python, and does not apply to ANY svm? Try to choose precise and fully appropriate tags, such as [tag:svm]. – Has QUIT--Anony-Mousse Dec 19 '13 at 11:50
-
Furthermore this kind of question is not related to programming but to maths / machine learning concepts. It should therefore better be asked on http://stats.stackexchange.com/ – ogrisel Dec 19 '13 at 18:29
3 Answers
11
The decision boundary for a linear support vector machine is an (affine) hyperplane.
For non-linear kernel support vector machines, the decision boundary of the support vector machine is not an hyperplane in the original feature space but a non-linear hypersurface (a surface of dimension n_features - 1) whose shape depends on the type of kernel.
However, the kernel function can be interpreted as inducing a non-linear mapping from the original feature space to some kernel space. In the kernel space then the decision function of the SVM is an hyperplane. Here is a video that gives an intuitive descriptions of the relation between the two for the polynomial kernel.
ogrisel
- 39,309
- 12
- 116
- 125
3
A decision boundary is a hypersurface that partitions the underlying vector space into two sets, one for each class. A general hypersurface in a small dimension space is turned into a hyperplane in a space with much larger dimensions.
Hyperplane and decision boundary are equivalent at small dimension space, 'plane' has the meaning of straight and flat, so it is a line or a plane that separate the data sets. When you do a non-linear operation to map your data to a new feature space, the decision boundary is still a hyperplane in that space, but is not a plane any more at the original space.
lennon310
- 12,503
- 11
- 43
- 61
-
So the hyperplane does not decide the separation between classes? – High schooler Dec 18 '13 at 23:42
-
hyperplane and decision boundary are equivalent at small dimension space, 'plane' has the meaning of straight, so it is a line or a plane that separate the data sets. When you do a non-linear operation to map your data to a high dimension, the decision boundary is still a hyperplane at that dimension, but is not a plane any more at the original data dimension. – lennon310 Dec 19 '13 at 00:07
-
The size of the dimension is not that important. It's the type of kernel (linear or not) that makes the decision boundary an hyperplane or not. – ogrisel Dec 19 '13 at 18:28
-
@ogrisel thank you for pointing out. I changed the word to 'space' to avoid the confusion. – lennon310 Dec 19 '13 at 18:31
2
When using kernel functions, the decision boundary will no longer remain a straight line.
(It will still be a hyperplane in a different, maybe infinite dimensional space, but this space will not actually be computed or used.)
Has QUIT--Anony-Mousse
- 76,138
- 12
- 138
- 194