0

I have an assignment:

Using Naive Bayes we built a model on some data with 2 classes (model returns 2 probabilities - one for positive and one for negative class). We calculated the area under ROC curve AUC = 0.8 and classification accuracy CA = 0.6 with threshold set to 0.5 (if the probability of some example for positive class is higher than 0.5, we predict positive class for that example, else the negative class). Then we discovered that if we set the threshold to 0.3, classification accuracy becomes CA = 0.7. What is the AUC for the second threshold? If the result depends on initial data, present all possibilities.

How can I calculate that?

peech
  • 931
  • 3
  • 12
  • 23

1 Answers1

0

Not sure if that qualifies as an answer, but the ROC AUC is the integral of sensitivity and specificity over all classification thresholds. Therefore you cannot compute the AUC for a specific threshold.

Calimo
  • 7,510
  • 4
  • 39
  • 61
  • That's the reason I can't answer the question, how do you construct ROC with only one threshold... I think I'm supposed to answer whether new AUC will be higher or lower given the previous AUC. By the way, this was a question on old exams from this course, so I doubt the correct answer is "cannot determine". – peech Feb 17 '15 at 09:51
  • When you construct a ROC curve, you use all thresholds in your classification. If you construct a ROC curve with a single threshold, then you're doing it wrong. – Calimo Feb 17 '15 at 10:29
  • I know. "how do you construct ROC with only one threshold..." was a retorical question :) – peech Feb 17 '15 at 10:39