Questions tagged [auc]

The area under the ROC curve can be thought of as a single scalar representation of the ROC curve itself. The AUC of a classifier has the property of being equivalent to the probability that the classifier will rank a randomly chosen positive data point higher than a randomly chosen negative data point.

The area under the ROC curve can be thought of as a single scalar representation of the ROC curve itself. Since this value represents part of the area of a 1x1 square, the AUC is a value between 0.0 and 1.0. However, since a classifier should always perform better than random, the realistic domain of the AUC values should be 0.5 to 1.0. The AUC of a classifier has the property of being equivalent to the probability that the classifier will rank a randomly chosen positive data point higher than a randomly chosen negative data point [Fawcett, 2006]. It can be shown that the AUC is related to the Gini coefficient [Hand et al, 2001]. The AUC can be estimated using trapezoidal approximation by considering the interval between consecutive points [Hand et al, 2001]

Fawcett, Tom. 2006. “An Introduction to ROC Analysis.” Pattern Recognition Letters 27 (8) (June): 861–874. doi:10.1016/j.patrec.2005.10.010.

Hand, David J, and Robert J Till. 2001. “A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems.” Machine Learning 45 (2) (January 1): 171–186. doi:10.1023/A:1010920819831.

535 questions
0
votes
1 answer

Getting a good cross validation score but a very bad Roc_auc score

I am very new to this, so any kind of information would be helpful. Apologies, if I have asked a very trivial question. I am working on a medium sized dataset with a lot of zeros. We have applied a lot of models and cv-skf score for k=10 has crossed…
Singh
  • 37
  • 1
  • 6
0
votes
1 answer

wrong AUC when using caTools in R

I was using the caTools package in R to get AUC in R. Only now I found out that it returns the same AUC when the probabilities are flipped. This is not really correct (see example below). Is there any theoretical reason where this makes…
Krrr
  • 452
  • 1
  • 3
  • 15
0
votes
0 answers

How to find AUC in python?

I have two files: predictions.csv and target.csv. Format of predictions.csv: SampleID,Target t1,-1.0454370703147253e-05 t2,-0.48161680725663214 t3,8.1420547483708091e-06 . . . t4950,-6.4382307796971309e-05 Format of target.csv: #SampleID,Target [0…
Black Dragon
  • 147
  • 1
  • 2
  • 13
0
votes
0 answers

How to obtain auc, specificity and sensitivity for a given cutoff value in R?

Suppose I have the data below set.seed(4) ctt <- runif(50,0,10) status <- rbinom(50,1,0.7) # 0 control, 1 patient and subjects with ctt >4.5 are normal (false positive) and ctt < 4.5 are abnormal (false negative). In this case, how could I obtain…
Günal
  • 751
  • 1
  • 14
  • 29
0
votes
1 answer

Compute AUC of a random uniform forest in regression

I have a dataframe df df<-structure(list(ID = structure(c(1L, 1L, 1L, 2L, 2L, 3L, 3L, 3L, 3L, 3L, 7L, 7L, 7L, 7L, 7L, 7L, 7L, 7L, 8L, 8L), .Label = c("AU-Tum", "AU-Wac", "BE-Bra", "BE-Jal", "BR-Cax", "BR-Sa3", "CA-Ca1", "CA-Ca2", "CA-Ca3",…
SimonB
  • 670
  • 1
  • 10
  • 25
0
votes
1 answer

Custom AUC in R with different thresholds and binary predictions

I am looking to plot a FPR vs TPR point on an AUC graph for different thresholds. For example, if data$C2 is my data frame with the true response column (either 0 or 1), I want to make a vector with predicted values (0 or 1) when data$C1 (a…
user2324
  • 3
  • 1
  • 4
0
votes
1 answer

AUC package - AUC error - r programming

I am trying to get a AUC plot working using the AUC package in R. I am unsure of the error and new to this fit is the trained model: test is the test data test$going_to_cross <- predict(fit, test, type="prob") prediction <- predict(fit, test,…
tony
  • 1,147
  • 2
  • 7
  • 10
0
votes
0 answers

R AUC with ROCR from Biomod2

I have to get AUC (area under the ROC curve) to evaluate my models. I am using the SRE model with Biomod2 package. Biomod2 gives me AUC value through evaluate() function which is nice. Then I tried to check AUC with ROCR package, and I got a big…
Spes Alpha
  • 27
  • 1
  • 9
0
votes
2 answers

Random forest evaluation in R

I am a newbie in R and I am trying to do my best to create my first model. I am working in a 2- classes random forest project and so far I have programmed the model as follows: library(randomForest) set.seed(2015) randomforest <-…
WillieM
  • 45
  • 1
  • 3
0
votes
2 answers

Statistical comparison of machine learning algorithm

I am working in machine learning. I am stuck in one of the thing. I want to compare 4 machine learning techniques among 10 datasets. After performing experiment i got Area Under Curve value. After this i have applied Analysis of variance test…
0
votes
2 answers

ROC Curve is convex

I am doing a ROC plot (and AUC calculation) of default frequencies, using logistic regression with one multi-class classifier 'sub_grade.' Assume lcd is a dataframe containing the initial data. Xtrain, Xtest, ytrain, ytest = …
GPB
  • 2,395
  • 8
  • 26
  • 36
0
votes
1 answer

Using the integrate function to find area under a continuous probability curve -finding % completion of a marketing campaign?

I'm trying to build a model to forecast direct mail marketing campaign responses. In the code below I was able to use responses from a previous campaign to create a smooth curve (i.e. continuous probability). Now, I need to find the total area under…
Ryan Chase
  • 2,384
  • 4
  • 24
  • 33
0
votes
1 answer

Calculate the area under the curve with different starting and end points

I want to calculate the area under the curve for each column of a big data frame in which the limits for the integration are different in each column and are stored in a different data frame (df2). Particularly, df2 indicates the position of the…
0
votes
1 answer

Calculating the area under curve from classification accuracy

I have an assignment: Using Naive Bayes we built a model on some data with 2 classes (model returns 2 probabilities - one for positive and one for negative class). We calculated the area under ROC curve AUC = 0.8 and classification accuracy CA = 0.6…
peech
  • 931
  • 3
  • 12
  • 23
0
votes
0 answers

compute AUC metric for Matrix Factorization output

Initial Matrix contains 1 for known links (0 for unknown and missed entries) in a link prediction system. Outputs of Matrix Factorization are predicted values for missed entries to calculate AUC: Hide 20% of known links (set 20% of entries with 1…
nourani
  • 71
  • 1
  • 6