-1

I have a dataset that is divided into training and test parts. My task is to train it and evaluate my model using k-fold cross validation. I'm a bit confused with the task statement. As far as I know the point of k-dold cross validation is to evaluate the model on a limited data sample by using all the data to train and test it. Please tell me if I'm correct in my algorithm:

  1. Concatenate the test and the train parts of my dataset.
  2. Split the whole dataset into k parts.
  3. Use the MLP to train my model on all parts of my dataset except one, then test it on the left part. Retain the evaluation score.
  4. Calculate the average evaluation score.
Kosh
  • 960
  • 2
  • 13
  • 28

1 Answers1

2

Yes you are doing it right. The whole point of using K-fold cross validation is because we have limited data and it ensures that every observation from the original dataset has the chance of appearing in training and test set.

Steps as you mentioned :

  1. Split the entire data randomly into k folds (value of k shouldn’t be too small or too high, ideally we choose 5 to 10 depending on the data size).

  2. Then fit the model using the K — 1 folds and validate the model using the remaining Kth fold. Save score and errors.

  3. Repeat this process until every K-fold serve as the test set. Then take the average of your recorded scores. That will be the performance metric for the model.

Edit for point 1: The higher value of K leads to less biased model but large variance might lead to overfit, where as the lower value of K is similar to the train-test split approach. Hence we choose k value to be between 5 to 10. You can experiment with these values to get better performance metric.