Questions tagged [auc]

The area under the ROC curve can be thought of as a single scalar representation of the ROC curve itself. The AUC of a classifier has the property of being equivalent to the probability that the classifier will rank a randomly chosen positive data point higher than a randomly chosen negative data point.

The area under the ROC curve can be thought of as a single scalar representation of the ROC curve itself. Since this value represents part of the area of a 1x1 square, the AUC is a value between 0.0 and 1.0. However, since a classifier should always perform better than random, the realistic domain of the AUC values should be 0.5 to 1.0. The AUC of a classifier has the property of being equivalent to the probability that the classifier will rank a randomly chosen positive data point higher than a randomly chosen negative data point [Fawcett, 2006]. It can be shown that the AUC is related to the Gini coefficient [Hand et al, 2001]. The AUC can be estimated using trapezoidal approximation by considering the interval between consecutive points [Hand et al, 2001]

Fawcett, Tom. 2006. “An Introduction to ROC Analysis.” Pattern Recognition Letters 27 (8) (June): 861–874. doi:10.1016/j.patrec.2005.10.010.

Hand, David J, and Robert J Till. 2001. “A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems.” Machine Learning 45 (2) (January 1): 171–186. doi:10.1023/A:1010920819831.

535 questions
4
votes
1 answer

how to use sklearn AUC in tensorflow.keras model metrics?

I'm trying to use sklearn AUC in tf.keras as model metrics , for that I used custom made function from this link AUC Below is my model : def auc(y_true, y_pred): return tf.py_func(roc_auc_score, (y_true, y_pred), tf.double) model =…
Arjunsai
  • 176
  • 1
  • 7
4
votes
0 answers

different output for PR AUC for different R packages

I find different numeric values for the computation of the Area Under the Precision Recall Curve (PRAUC) with the dataset I am working on when computed via 2 different R packages: yardstick and caret. I am afraid I was not able to reproduce this…
deltasun
  • 314
  • 3
  • 11
4
votes
3 answers

ROC curve for Isolation Forest

I am trying to plot the ROC curve to evaluate the accuracy of Isolation Forest for a Breast Cancer dataset. I calculated the True Positive rate (TPR) and False Positive Rate (FPR) from the confusion matrix. However, I do not understand how the TPR…
Nnn
  • 191
  • 3
  • 9
4
votes
3 answers

Computing AUC in SQL

What's the best way to compute AUC in SQL? Here is what I got (assuming table T(label, confid) and label=0,1): SELECT sum(cumneg * label) * 1e0 / (sum(label) * sum(1-label)) AS auc FROM ( SELECT label, sum(1-label) OVER(ORDER BY confid ROWS…
iggy
  • 662
  • 6
  • 14
4
votes
1 answer

Calculate AUC manually without using any Numpy or Sklearn library

I have given a set of X, Y coordinate and I need to find the AUC using trapezoidal formula, without using any numpy or sklearn library. (x0,y0) is always (0,0) (xn,yn) is always (1,1) Below diagram Without using any sklearn library, I understand…
nad
  • 2,640
  • 11
  • 55
  • 96
4
votes
1 answer

R - ROC Curves/AUC Specificity vs 1-Specificity

I have created a few predictive models and I am in the process of evaluating them by looking at the ROC Curve and AUC. Currently, I have Specificity on X axis, however, when I researched ROC Curves, I saw 1 - Specificity on the X axis. What is the…
MhQ-6
  • 328
  • 3
  • 11
4
votes
4 answers

Find midpoints between values in vector

This is for a function to calculate the AUC using the Midpoint Rule. Using R how can I define a vector that contains the midpoints between values of a previous vector? Or how can I shift the values of a vector to their midpoints? # define h (or…
abk
  • 309
  • 6
  • 15
4
votes
1 answer

Why does my ROC curve look like a V?

newpred <- c(1, 0 ,0 ,1 ,0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 1, 0, 0, 1, 0, 1, 0, 0, 1, 1, 1, 0,0, 1, 0, 0, 0, 0,0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 1, 0, 1, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 1, 1, 1, 0, 0,…
L.Sobble
  • 49
  • 1
  • 10
4
votes
0 answers

scikit learn high test set AUC but low training set Cross validates AUC

I am unclear on how my test set AUC can be so consistently high, but my training set cross validated AUC 'roc_auc' can be so much lower. The more usual situation is the reverse (high training set CV, low test set) due to over-fitting. Why might my…
ivan7707
  • 1,146
  • 1
  • 13
  • 24
4
votes
1 answer

grid search with weighted AUC

So I am using the sample weights function and I don't want to measure performance using the default sklearn scoring function. It looks like it says Here that I can pass GridSearchCV the string 'roc_auc' and it should calculate auc for me but will…
Chris
  • 12,900
  • 12
  • 43
  • 65
4
votes
2 answers

How to calculate AUC for One Class SVM in python?

I have difficulty in plotting OneClassSVM's AUC plot in python (I am using sklearn which generates confusion matrix like [[tp, fp],[fn,tn]] with fn=tn=0. from sklearn.metrics import roc_curve, auc fpr, tpr, thresholds = roc_curve(y_test,…
imkhan
  • 171
  • 4
  • 16
3
votes
2 answers

AUC of logistic and ordinal model following multiple imputation using MICE (with R)

I am asking a question concerning the additive predictive benefit of the inclusion of a variable to a logistic and an ordinal model. I am using mice to impute missing covariates and am having difficulty finding ways to calculate the AUC and R…
DW1310
  • 147
  • 7
3
votes
1 answer

SVM performance not consistent with AUC score

I have a dataset that contains information about patients. It includes several variables and their clinical status (0 if they are healthy, 1 if they are sick). I have tried to implement an SVM model to predict patient status based on these…
A Jorg
  • 141
  • 1
  • 5
3
votes
2 answers

Plotting the ROC curve for a multiclass problem

I am trying to apply the idea of sklearn ROC extension to multiclass to my dataset. My per-class ROC curve looks find of a straight line each, unline the sklearn's example showing curve's fluctuating. I give an MWE below to show what I mean: # all…
user12587364
3
votes
1 answer

What's the difference between Keras' AUC(curve='PR') and Scikit-learn's average_precision_score?

I am quite confused on the difference between Keras' AUC(curve='PR') and Scikit-learn's average_precision_score. My objective is to compute the Area Under the Precision-Recall Curve (AUPRC), for both Scikit-learn and Keras models. However, these two…
Stoner
  • 846
  • 1
  • 10
  • 30