0

I am using SuperLearner for repeated V-fold or cross validation. Is there a way to obtain individual results for each of the say V=10 folds?

# Packages
library(tidyverse)
library(SuperLearner)
library(caret)

# Some data
X <- matrix(rnorm(1000 * 50), nrow = 1000, ncol = 50)
Y <- X[, 1] + sqrt(abs(X[, 2] * X[, 3])) + X[, 2] - X[, 3] + rnorm(1000)
df <- cbind(Y, X) %>% as.data.frame()

# Cross validation in SuperLearner
model = CV.SuperLearner(Y = df$Y,
                        X = df %>% select(-Y), 
                        family = gaussian(),
                        SL.library = c("SL.lm"),
                        V = 5)

# Look up folds
model$folds

# Look up individual results for each fold...?

If you compare this to caret, it's pretty straight forward.

# Cross validation in caret
train.control <- trainControl(method = "repeatedcv", 
                              number = 5, 
                              repeats = 5)

model <- train(Y ~ ., 
               data = df, 
               method = "lm",
               trControl = train.control, 
               tuneLength = 10)

# Look up results
model$resample

Long story short, where is the model$resample in SuperLearner?

Dr. Fabian Habersack
  • 1,111
  • 12
  • 30

0 Answers0