3

I am computing a SVM-RFE model using "ROC" as the metric, with the rfe function (caret package). To my knowledge, the rfe algorithm optimizes the AUC values using the roc function of the pROC package, with its predefined arguments. However, I would like to set the direction argument to "<" instead of "auto" because in some cases the resulting average AUC is computed in reverse (my data is not very good...). This problem is explained in the answer here: Difference in average AUC computation using ROCR and pROC (R)

How could I change this default argument value of the roc function in the rfe computation?

I have tried this simple option, but it does not work:

svmRFE_NG3 <- rfe(x = TAll[,2:50],
               y = TAll[,1],
               sizes = seq(1,42),
               metric = "ROC",
               levels = c("BREAST","LUNG"),
               direction = "<",
               rfeControl = FSctrl,
               ## Options to train()
               method = "svmLinear",
               tuneLength = 10,
               preProc = c("center", "scale"),
               ## Inner resampling process
               trControl = TRctrl)
Community
  • 1
  • 1
Rafa OR
  • 339
  • 2
  • 3
  • 8

1 Answers1

2

I had a look at the source code of caret, and it doesn't seem to be readily possible at the moment. Arguments are not passed down to the call to the roc function.

I would suggest submitting a request for enhancement on the github repository of the package. Max Kuhn the maintainer is pretty responsive and you have a good chance to see this implemented in a future version.

Calimo
  • 7,510
  • 4
  • 39
  • 61
  • I know that I should avoid this type of comments here, but I have to thank you @Calimo for all your help because I am new with R and it is being difficult to deal with all these kind of problems I have! – Rafa OR May 25 '16 at 13:18