"The format of predictions is incorrect"

Question

Implementation of ROCR curve, kNN ,K 10 fold cross validation. I am using Ionosphere dataset.

Here is the attribute information for your reference:

-- All 34 are continuous, as described above -- The 35th attribute is either "good" or "bad" according to the definition summarized above. This is a binary classification task.

data1<-read.csv('https://archive.ics.uci.edu/ml/machine-learning-databases/ionosphere/ionosphere.data',header = FALSE)

knn on its own works, kNN with kfold also works. But when I put in the ROCR code it doesnt like it. I get the error: "The format of predictions is incorrect". I checked the dataframes pred and Class 1. The dimensions are same. I tried with data.test$V35 instead of Class1 I get the same error with this option.

 install.packages("class")
    library(class)

nrFolds <- 10
data1[,35]<-as.numeric(data1[,35])

# generate array containing fold-number for each sample (row)
folds <- rep_len(1:nrFolds, nrow(data1))


# actual cross validation
for(k in 1:nrFolds) {
  # actual split of the data
  fold <- which(folds == k)
  data.train <- data1[-fold,]
  data.test <- data1[fold,]

  Class<-data.train[,35]
  Class1<-data.test[,35]
  # train and test your model with data.train and data.test

  pred<-knn(data.train, data.test, Class, k = 5, l = 0, prob = FALSE, use.all = TRUE)
  data<-data.frame('predict'=pred, 'actual'=Class1)
  count<-nrow(data[data$predict==data$actual,])
  total<-nrow(data.test)
  avg = (count*100)/total
  avg =format(round(avg, 2), nsmall = 2)
  method<-"KNN" 
  accuracy<-avg
  cat("Method = ", method,", accuracy= ", accuracy,"\n")
}

install.packages("ROCR")
library(ROCR)
rocrPred=prediction(pred, Class1, NULL)
rocrPerf=performance(rocrPred, 'tpr', 'fpr')
plot(rocrPerf, colorize=TRUE, text.adj=c(-.2,1.7))

Any help is appreciated.

I don't see any indexing on the LHS of assignments inside that loop. That's a common type of R-newb error. (I'm also unsure if ROCR is set up to handle multi-fold CV challenges.) — IRTFM, Nov 04 '16 at 19:08
Thank you for the comment. Guilty- R-newb. I did not understand what you meant by indexing on LHS of assignments. Could you point me at the line of code you think is erroneous.. — codingyo, Nov 04 '16 at 19:30
I honestly don't know if it is "erroneous", but are you aware that every time you assign to `pred`, data`, `count`, `total`, `avg` that you are over-writing the prior value for each of those? If instead, you set them up as lists you could do: `pred[[i]] <- knn(...)` — IRTFM, Nov 04 '16 at 21:36

score 0 · Accepted Answer · answered Nov 09 '16 at 18:43

This worked for me..

install.packages("class")
library(class)
library(ROCR)
nrFolds <- 10
data1[,35]<-as.numeric(data1[,35])

# generate array containing fold-number for each sample (row)
folds <- rep_len(1:nrFolds, nrow(data1))


# actual cross validation
for(k in 1:nrFolds) {
  # actual split of the data
  fold <- which(folds == k)
  data.train <- data1[-fold,]
  data.test <- data1[fold,]

  Class<-data.train[,35]
  Class1<-data.test[,35]
  # train and test your model with data.train and data.test

  pred<-knn(data.train, data.test, Class, k = 5, l = 0, prob = FALSE, use.all = TRUE)
  data<-data.frame('predict'=pred, 'actual'=Class1)
  count<-nrow(data[data$predict==data$actual,])
  total<-nrow(data.test)
  avg = (count*100)/total
  avg =format(round(avg, 2), nsmall = 2)
  method<-"KNN" 
  accuracy<-avg
  cat("Method = ", method,", accuracy= ", accuracy,"\n")

  pred <- prediction(Class1,pred)
  perf <- performance(pred, "tpr", "fpr")
  plot(perf, colorize=T, add=TRUE)
  abline(0,1)
}

"The format of predictions is incorrect"

1 Answers1