0

I have never seen such learning curves. Am I right, that huge overfitting occurs? The model is fitting better and better to the training data, while it generalizes worse for the test data.

Usually when there is high variance, like here, more examples should help. In this case, they won't, I suspect. Why is that? Why such example of learning curves can't be found easily in literature/tutorials?

Learning curves. SVM, param1 is C, param2 is gamma

DavidS1992
  • 823
  • 1
  • 8
  • 19

1 Answers1

1

You have to remember that SVM is non parametric model, thus more samples does not have to reduce variance. Reduction in variance can be more or less guaranteed for parametric model (like neural net), but SVM is not one of them - more samples mean not only better training data but also more complex model. Your learning curves are typical example of SVM overfitting, which happens a lot with RBF kernel.

lejlot
  • 64,777
  • 8
  • 131
  • 164
  • Thank you for your answer. I do not have enough experience with non parametric model, so I did not think about it. Can you tell me, what if I have similar plot, but validation curve goes up, just like the training curve? Weired thing for me is that training set goes up all the time. Gap between curves is still big, say constant. Can I assume, that nothing has changed, the model still overfits? – DavidS1992 Jul 14 '16 at 09:34