0

I am trying to use multiclass.au1p measure in mlr package. It gave me an error saying

Error in FUN(X[[i]], ...) : Measure multiclass.au1p requires predict type to be: 'prob'!

When I tried to set the predict type to prob then it gave me an error similar to following for any classifier i used

Error in setPredictType.Learner(learner, predict.type) : Trying to predict probs, but classif.xgboost.multiclass does not support that!

How can I resolve this?

Following is my code

  trainTask <- makeClassifTask(data = no_out_pso,target = "response_grade")

  Clslearn = makeLearner("classif.xgboost", predict.type = "prob")
  Clslearn = makeMulticlassWrapper(Clslearn, mcw.method = "onevsrest")
  Clslearn = setPredictType(Clslearn, "prob")

  rdesc = makeResampleDesc("CV", iters = 3)

  r = resample(Clslearn, trainTask, rdesc, measures = list(mlr::acc, mlr::multiclass.au1p, mlr::multiclass.au1u))

  print(r)
Roshana
  • 357
  • 1
  • 3
  • 17

2 Answers2

4

It does not work with the makeMulticlassWrapper, because this does not support probability prediction (at the moment). I get also an error, when I try to set it to prob in your code.

Code that works: 

Clslearn = makeLearner("classif.xgboost", predict.type = "prob")
rdesc = makeResampleDesc("CV", iters = 3)
r = resample(Clslearn, iris.task, rdesc, measures = list(mlr::acc, mlr::multiclass.au1p, mlr::multiclass.au1u))
PhilippPro
  • 659
  • 4
  • 12
2

You need to use a classifier that supports predicting probabilities. You can get a list with the listLearners() function:

listLearners(properties = "prob")
Lars Kotthoff
  • 107,425
  • 16
  • 204
  • 204
  • hey but when I tried listLearners("classif", properties = c("multiclass", "prob"))[c("class","multiclass","prob") ] xgboost and the other learners i tried, returned true for both mullticlass and prob – Roshana Apr 13 '17 at 07:27