0

Is there any way of choosing the best threshold while computing the confusion matrix so that number of false positives and false negatives get minimized (to get the best possible combination)?

Till now I have tried changing values of threshold manually but that is too exhausting.

Thanks in advance.

Richie Cotton
  • 118,240
  • 47
  • 247
  • 360
user3664020
  • 2,980
  • 6
  • 24
  • 45
  • What did you try yet? – ChrKoenig Aug 21 '14 at 11:17
  • manually changing the threshold to different values. But that is too exhausitng. I could not find anything also on Google. Do you have any solution? – user3664020 Aug 21 '14 at 11:30
  • Not really. I actually wanted to incite you to give some more information about your problem and what you've tried yet. Usually people don't put effort in answering questions that consist of only one sentence. – ChrKoenig Aug 21 '14 at 11:45
  • 1
    You may need to use ROC curve. You may get more help on http://stats.stackexchange.com/ (CrossValidated). – rnso Aug 21 '14 at 12:22
  • 2
    Use the ROC curve; that's what it was create for. Also, see [this page](http://topepo.github.io/caret/custom_models.html#Illustration5) to automatically optimize it. – topepo Aug 21 '14 at 12:56

2 Answers2

0

you can use WEKA to optimize the threshold. The optimal threshold will be reached, when the accuracy of the model is the highest. Otherwise you can also use the ROC curve

josef
  • 1
-1

I copied this code from somewhere to get the optimal threshold. I think it gives what is asked in here:

pred <- prediction(data$predicted_values, data$observations)
ss <- performance(pred, "sens", "spec")
plot(ss)
best_threshold <- ss@alpha.values[[1]][which.max(ss@x.values[[1]]+ss@y.values[[1]])]

Hope it can help

Carlos Bautista León
  • 83
  • 6