0

How can I apply backwards variable selection based on performance on test data in R?

I already know that there is the stepAIC() function which does almost what i want, but in every step it removes one variable based on the AIC criteria. i want to do the same with test and training data and remove the variable in every step based on prediction performance.

Joshua_ABC
  • 13
  • 3
  • you're probably going to have to write this yourself ... – Ben Bolker Jun 04 '23 at 13:32
  • Really? But how is it possible, I'm the first one that wants to do that? is there sth wrong about this approach? – Joshua_ABC Jun 04 '23 at 13:34
  • I can't say for sure that it *doesn't* exist. In general AIC is a pretty good approximation of out-of-sample performance (asymptotically, AIC is equivalent to leave-one-out cross-validation: https://stats.stackexchange.com/questions/322537/if-the-aic-and-the-bic-are-asymptotically-equivalent-to-cross-validation-is-it ) – Ben Bolker Jun 04 '23 at 13:38
  • So (continuing) I think people don't generally bother. Either they would filter on AIC or adjusted R^2 or something, *or* they would use something like LASSO to select (wrappers vs embedded methods, e.g. see https://sebastianraschka.com/faq/docs/feature_sele_categories.html ) – Ben Bolker Jun 04 '23 at 13:57

0 Answers0