"Something is wrong; all the Accuracy metric values are missing" while using partykit, caret, recipes

Question

I am trying to train around 15 machine learning models, using recipes (for consistent pre-processing) and caret (for consistent training). The only 2 models that consistently give me the error "Something is wrong; all the Accuracy metric values are missing" are in the partykit package -- cforest and ctree. Below I show the error using the PimaIndiansDiabetes dataset from mlbench.

my_rec <- recipe(diabetes ~ ., data = PimaIndiansDiabetes) %>%
  step_dummy(all_nominal(), -diabetes)%>%
  step_nzv(all_predictors())

fitControl5 <- trainControl(summaryFunction = twoClassSummary, 
                             verboseIter = TRUE, 
                             savePredictions =  TRUE, 
                             sampling = "smote", 
                             method = "repeatedcv", 
                             number= 5, 
                             repeats = 1,
                             classProbs = TRUE)

dtree5 <- train(my_rec, data = PimaIndiansDiabetes,
                 method = "cforest",
                 metric = "Accuracy",
                 tuneLength = 8,
                 trainControl = fitControl5)

note: only 7 unique complexity parameters in default grid. Truncating the grid to 7 .

Something is wrong; all the Accuracy metric values are missing:
    Accuracy       Kappa    
 Min.   : NA   Min.   : NA  
 1st Qu.: NA   1st Qu.: NA  
 Median : NA   Median : NA  
 Mean   :NaN   Mean   :NaN  
 3rd Qu.: NA   3rd Qu.: NA  
 Max.   : NA   Max.   : NA  
 NA's   :7     NA's   :7    
Error: Stopping
In addition: There were 50 or more warnings (use warnings() to see the first 50)

Below is code for method ctree

dtree6 <- train(my_rec, data = PimaIndiansDiabetes,
                 method = "ctree",
                 metric = "Accuracy",
                 tuneLength = 8,
                 trainControl = fitControl5)
Something is wrong; all the Accuracy metric values are missing:
    Accuracy       Kappa    
 Min.   : NA   Min.   : NA  
 1st Qu.: NA   1st Qu.: NA  
 Median : NA   Median : NA  
 Mean   :NaN   Mean   :NaN  
 3rd Qu.: NA   3rd Qu.: NA  
 Max.   : NA   Max.   : NA  
 NA's   :8     NA's   :8    
Error: Stopping
In addition: There were 50 or more warnings (use warnings() to see the first 50)

I would really appreciate your help!

using library(mlbench), I still get error: Error: object 'PimaIndiansDiabetes' not found — Oscar Kjell, Dec 22 '20 at 18:04

score 0 · Answer 1 · answered Jan 04 '21 at 03:32

The argument should be trControl = instead of trainControl = . If i run the below it works:

dtree5 <- train(my_rec, data = PimaIndiansDiabetes,
                  method = "cforest",
                  metric = "Accuracy",
                  tuneLength = 3,
                  trControl = fitControl5)

The output:

dtree5
Conditional Inference Random Forest 

768 samples
  8 predictor
  2 classes: 'neg', 'pos' 

Recipe steps: dummy, nzv 
Resampling: Cross-Validated (5 fold, repeated 1 times) 
Summary of sample sizes: 614, 615, 614, 615, 614 
Addtional sampling using SMOTE

Resampling results across tuning parameters:

  mtry  ROC        Sens   Spec     
  2     0.8298281  0.788  0.7013277
  5     0.8256038  0.794  0.7013277
  8     0.8222572  0.798  0.7276031

ROC was used to select the optimal model using the largest value.
The final value used for the model was mtry = 2.

"Something is wrong; all the Accuracy metric values are missing" while using partykit, caret, recipes

Below is code for method ctree

1 Answers1