I am using svm light to train a model for binary classification. Using the model, I tested some examples. I was surprised to see the output of the prediction file, it contains values greater than 1 as well as less than -1. I thought the range is [-1,1]. Am I doing something wrong?
Asked
Active
Viewed 680 times
1
-
Why whould you have a range [-1;1]? If it is for classification with two class, you should have {-1;1}. Some variants give a probability within ]-1;1[ when the new data is in the gutter. Others look for the distance to the gutter, thus having value different from {-1;1} outside it. Would you mind explaining a bit more what you expect and why? – Igor OA Jul 22 '16 at 07:52
-
@IgorOA, my understanding was the prediction is -1 and +1 for negative and positive example respectively. But the prediction may be any value between -1 and 1, as a measure of certainty (-1/+1 means 100% certain). But this seems wrong. – Rakib Jul 22 '16 at 18:05
1 Answers
2
It makes sense why the values are not bounded by the interval of [-1, 1] if you understand how the SVM works. An SVM tries to draw the line which separates the negative and positive data points while maximizing their distances from the line.
The values in the prediction file represent the distances of data from the SVM optimal hyperplane, where positive values are on the positive class side of the hyperplane and negative values are on the negative class side of the hyperplane. These distance can be arbitrarily large or small and are not bounded as can be seen by this image:
I've seen some SVM implementations such as Weka's implementation of Platt's SMO which normalize the values so that they are confidence values on the positive class bounded by the interval of [0, 1], but both ways work just fine for determining how confident an SVM is on a classification since a data point further from the hyperplane is more certain than one lying close to the hyperplane.
Tomer Aberbach
- 536
- 3
- 11