Calculating AUC ratio in R

Question

I am generating ecological niche models for a set of species and I would like to use AUC as a metric for ecological niche quality. Steven Phillips, who developed Maxent, provides code in his Maxent manual for calculating the AUC in R. However, I am reading papers that report partial AUC ratios as a more robust and conceptually sound metric. I think I understand how to calculate partial AUC using the ROCR R package, but how does one calculate AUC ratio?

Here is the tutorial script from Phillips:

presence<-read.csv("bradypus_variegatus_samplePredictions.csv")
background<-read.csv("bradypus_variegatus_backgroundPredictions.csv")
pp<-presence$Logistic.prediction
testpp<-pp[presence$Test.or.train=="test"]
trainpp<-pp[presence$Test.or.train=="train"]
bb<-background$logistic

combined<-c(testpp,bb)
label<-c(rep(1,length(testpp)),rep(0,length(bb)))
pred<-prediction(combined,label)
perf<-performance(pred,"tpr","fpr")
plot(perf,colorize=TRUE)
performance(pred,"auc")@y.values[[1]] #RETURNS AUC

AUC<-function(p,ind){
    pres<-p[ind]
    combined<-c(pres,bb)
    label<-c(rep(1,length(pres)),rep(0,length(bb)))
    predic<-prediction(combined,label)
    return(performance(predic,'auc')@y.values[[1]])
}

b1<-boot(testpp,AUC,100) #RETURNS AUC WITH STANDARD ERROR
b1

Any advice or suggestions would be greatly appreciated! Thank you.

score 2 · Accepted Answer · answered Oct 19 '11 at 04:03

2

Without knowing the specifics of your dataset and application,

Partial AUC: The area under only a portion of the curve. (usually picked because it is more robust or otherwise desirable, like you said)
AUC ratio: The ratio of one AUC to another. (usually a reference of some sort)

Soo...

Partial AUC ratio: The ratio of one partial AUC to another.

answered Oct 19 '11 at 04:03

John Colby

22,169
4
57
69

So are you saying that AUC ratio is not really a measure of quality of one niche model, but more of a metric when comparing several modeling outputs with some certain parameter varying? – Pascal Oct 19 '11 at 04:26
Correct, as I see it used generally. It's still possible there is some more specific meaning in your field, however. I don't know anything about ecological niche models, so unfortunately I'm no help there. Anyway, here's an example of how I typically see it used, in case it helps: http://www.drug-interactions.eu/eng/AUC.htm – John Colby Oct 19 '11 at 04:34
Ok, you're probably right. I think I may have been misunderstanding what they were trying to show with AUC ratios in the paper I was reading. Thanks for the clarification! – Pascal Oct 19 '11 at 04:50

score 1 · Answer 2 · answered Oct 19 '11 at 07:06

1

Package ROCR can calculate partial AUC values using the fpr.stop= parameter. As John said the ratio is just this value divided by the same calculation for your reference model.

answered Oct 19 '11 at 07:06

Allan Engelhardt

1,421
10
5

Calculating AUC ratio in R

2 Answers2