5

I have this data on houses from the Kaggle practice competition and I'm using rpart to train a simple first model to predict the sale price.

The model is not correctly identifying sales where the sale condition was abnormal or a down payment. Therefore, I'd like to increase the importance of this variable which is obviously overlooked in the model.

I'm assuming this is done by using the "weights" parameter but how is this parameter used? How can I pinpoint which variables I want to have a higher weight?

GreenManXY
  • 401
  • 1
  • 5
  • 14
  • 3
    the weights in `rpart` are *case weights*. They weight the observations, not the features. – Zelazny7 Apr 17 '17 at 13:17
  • How to add weight to a feature then? Just copy the observations? – GreenManXY Apr 17 '17 at 13:32
  • I don't know of any algorithms that provide weighted feature selection. Although I'm sure they exist. I would take the output of your full decision tree and use it as an input to a second decision tree with your sale condition feature. This would build a second tree with just two features. If it doesn't come in, then it's probably accounted for in another feature. – Zelazny7 Apr 17 '17 at 13:40
  • I'll try and play around with it, thanks. – GreenManXY Apr 17 '17 at 14:55

1 Answers1

5

From the documentation:

weights

optional case weights.

cost

a vector of non-negative costs, one for each variable in the model. Defaults to one for all variables. These are scalings to be applied when considering splits, so the improvement on splitting on a variable is divided by its cost in deciding which split to choose.

The weights is for rows (e.g. give higher weight to smaller class), the cost is for columns.

Example usage for applying the weights parameter (not necessarily the best way to define the weights):

positiveWeight = 1.0 / (nrow(subset(training, Y == TRUE)) / nrow(training))
negativeWeight = 1.0 / (nrow(subset(training, Y != TRUE)) / nrow(training))

modelWeights <- ifelse(training$Y== TRUE, positiveWeight, negativeWeight)

dtreeModel <- rpart(predFormula, training, weights = modelWeights)
Danny Varod
  • 17,324
  • 5
  • 69
  • 111