Using LIBSVM grid.py for unbalanced data?

Question

I'm having a three class problem with unbalanced data (90%, 5%, 5%). Now I want to train a classifier using LIBSVM.

The problem is that LIBSVM optimizes its parameter gamma and Cost for optimal accuracy, which means that 100% of the examples are classified as class 1, which is of course not what I want.

I've tried modifying the weight parameters -w without much success.

So what I want is, modifying grid.py in a way that it optimizes Cost and gamma for precision and recall separated by classes rather than for overall accuracy. Is there any way to do that? Or are there other scripts out there that can do something like this?

Oriol Nieto · Accepted Answer · 2012-07-11T13:26:39.780

8

The -w parameter is what you need for unbalanced data. What have you tried so far?

If your classes are:

class 0: 90%
class 1: 5%
class 2: 5%

You should pass the following params to svm:

-w0 5 -w1 90 -w2 90

edited Jul 11 '12 at 13:26

answered Jul 10 '12 at 15:08

Oriol Nieto

5,409
6
32
38

1

thanks, but I think it should be the other way round: -w0 5 -w1 90 -w2 90, since the smaller class should have more costs associated with them.. this one helped! – Damnum Jul 11 '12 at 10:25
yeah, I think you're right. I just edited my question. Thanks! – Oriol Nieto Jul 11 '12 at 13:26
And when you have more than 3 classes, how can you attribute the value of each w ? – lilouch Jan 11 '17 at 13:50

score 4 · Answer 2 · answered Jul 14 '12 at 13:09

4

If you want to try an alternative, one of the programs in the svmlight family, http://www.cs.cornell.edu/people/tj/svm_light/svm_rank.html, directly minimizes the area under the ROC curve.

Minimizing the AUC may give better results than re-weighting training examples.

answered Jul 14 '12 at 13:09

user1149913

4,463
1
23
28

1

svmlight is commercial-unfriendly; it's only free for academic use. – JDonner Mar 20 '13 at 23:14

score 0 · Answer 3 · answered Mar 06 '17 at 13:54

0

You can optimize any of the precision, recall, F-score and AUC using grid.py. Tweak is that you have to change cross validation evaluation measure used by svm-train in LIBSVM. Follow the procedure given on LIBSVM website.

answered Mar 06 '17 at 13:54

dnivog

123
9

score 0 · Answer 4 · answered Mar 22 '17 at 15:51

If you have unbalanced data, you probably shouldn't be optimizing accuracy. Instead optimize f-score (or recall, if that's more important to you). You can change the evaluation function as described here.

I think you should also optimize gamma and Cost, while using different class weight configurations. I modified the "get_cmd" function in grid.py by passing different class weights for that purpose (-wi weight). In my experience, class weighting doesn't always help.

Using LIBSVM grid.py for unbalanced data?

4 Answers4