does we need significant test when we use 10-fold cross validation?

Question

Usually to show that our results are not by chance we use significant test like t-test. But when we use 10-fold cross validation we learn&test our modals over chunks of dataset. I'm wondering does we need t-test when we have used 10-fold cross validation? To be more precise I mean when we should use 10-fold cross validation?

You might have more luck asking this at the [Stats Stackexchange](http://stats.stackexchange.com/) — Lewis Norton, Jun 05 '15 at 13:08
I find it hard to find any similarities between those two concepts. — cel, Jun 05 '15 at 13:11
@amirveyseh, it would be nice if you could add some more details about what you think is similar. To me the question looks a little like asking about the differences between a plane and a tree. It's not really possible to give a reasonable answer to that. — cel, Jun 05 '15 at 13:16
Ohh....Ok I'll express my questions in a more clear an appropriate manner. Thanks — user3070752, Jun 05 '15 at 13:18

score 1 · Answer 1 · answered Jun 05 '15 at 14:37

t-test, when applied to the difference of two normally-distributed variables, is a parametric test that estimates the confidence we have in the fact that the two variables have distinct expectations.

10-fold cross validation is a resampling technique that can be used to estimate confidence we can have in a statistics (e.g., but not only, the difference between two random variables), when we do not know the distribution of the statistics in advance (non-parametric settings).

With cross-validation, you do not get nice 'it is significant' results, you simple have a distribution of the estimations of that statistics, of which the variance is an indicator of the confidence we can have.

Cross validation is often used in model selection when building non-parametric regression or classification models.

score 1 · Accepted Answer · answered Jun 06 '15 at 05:10

T-test is a type of statistical test on your data. Say you are comparing two datasets and you want to know if the two data sets are significantly different from each other. Then you will do a t-test.

Cross validation is more of a technique for evaluation your models. Typically used to ensure that your model is not overfitting.You do this while building your model to ensure you have a model that is able to generalize to future data. There is not any kind of a statistical test being used here. I hope this helps your doubt.

does we need significant test when we use 10-fold cross validation?

2 Answers2