3

I have used caret library to compute class probabilities and predictions for a binary classification problem, using 10-fold cross validation and 5 times repetition.

Now I have TRUE (observed values for each data point) values, PREDICTED (by an algorithm) values, Class 0 probabilities and Class 1 probabilities which were used by an algorithm to predict class label.

Now how can I create an roc object using either ROCR or pROC library and then calculate auc value?

Assume that I have all these values stored in predictions dataframe. e.g. predictions$pred and predictions$obs are the predicted and true values respectively, and so on...

milos.ai
  • 3,882
  • 7
  • 31
  • 33
exAres
  • 4,806
  • 16
  • 53
  • 95

1 Answers1

5

Since you did not provide a reproducible example, I'm assuming you have a binary classification problem and you predict on Class that are either Good or Bad.

predictions <- predict(object=model, test[,predictors], type='prob')

You can do:

> pROC::roc(ifelse(test[,"Class"] == "Good", 1, 0), predictions[[2]])$auc
# Area under the curve: 0.8905
Steven Beaupré
  • 21,343
  • 7
  • 57
  • 77