21

The help page for randomforest::randomforest() says:

"classwt - Priors of the classes. Need not add up to one. Ignored for regression."

Could setting the classwt parameter help when you have heavy unbalanced data, ie. priors of classes differs strongly ?

How should I set classwt when training a model on a dataset with 3 classes with a vector of priors equal to (p1,p2,p3), and in test set priors are (q1,q2,q3)?

crazybilly
  • 2,992
  • 1
  • 16
  • 42
Qbik
  • 5,885
  • 14
  • 62
  • 93
  • 3
    I'm not sure about your second question, but `classwt` I believe is used when sampling from your data, such that each sample for each tree is drawn from your classes with those probabilities (after normalization). – joran Apr 11 '12 at 22:45
  • strata yields the same performance as classwt and is easier to control, check out this guide on SO: http://stats.stackexchange.com/questions/157714/r-package-for-weighted-random-forest-classwt-option/158030#158030 – Soren Havelund Welling Nov 20 '15 at 09:53
  • @joran that's stratafication your're describing. Classwt is used to weight samples when to find optimal break point for node split and to define node prediction during training. – Soren Havelund Welling Nov 20 '15 at 09:56
  • may my old comments also haunt me four years later :) – Soren Havelund Welling Nov 20 '15 at 09:58

1 Answers1

23

could setting classwt parameter help when you have heavy unbalanced data - priors of classes differs strongly?

Yes, setting values of classwt could be useful for unbalanced datasets. And I agree with joran, that these values are trasformed in probabilities for sampling training data (according Breiman's arguments in his original article).

How set classwt when in training dataset with 3 classes you have vector of priors equal to (p1,p2,p3), and in test set priors are (q1,q2,q3)?

For training you can simply specify

rf <- randomForest(x=x, y=y, classwt=c(p1,p2,p3))

For test set no priors can be used: 1) there is no such option in predict method of randomForest package; 2) weights have only sense for training of the model and not for prediction.

DrDom
  • 4,033
  • 1
  • 21
  • 23
  • As I understand priors (p1,p2,p3) are characteristic of general population, not the specific training dataset. If I want to predict classes in the test dataset and I know that classes probabilies in the set are (q1,q2,q3) than setting classwt=c(q1,q2,q3) should help random forest to explore training space in better way. – Qbik Apr 14 '12 at 08:07
  • No, these class weights are specific for the training set only. For example, if you have balanced training set, in general there is no need to use `classwt` parameter. But at the same time you can have unbalanced test set, and I expect that changing class weghts will not improve test set prediction in such a case. In another words, using `classwt` you may increase prediction accuracy for one of the classes and simultaneously descrease for another. You can play with this parameter a bit and look on OOB set prediction statistics, for example. – DrDom Apr 14 '12 at 10:45