I'm looking at using h2o's AutoML functionality to benchmark different model algorithms, but I wish to do it with a custom cross-validation strategy. Based on the current documentation, I understand that AutoML default CV method is the conventional K-Fold CV.
However, I'm looking at performing a Forward next day CV method to replicate daily retraining of the data. For example, assuming I have 100 days of data:
- For the first iteration, I will train it from Day 1 to Day 80, and score the prediction for Day 81
- For the 2nd iteration, I will train it from Day 1 to 81, and score the prediction for Day 82
- This process is repeated for all remaining days, and the validation score is based on the predictions from Day 81 to 100.
Is it possible to do that in either the Python or R version of h2o's package?
Thanks in advance!
