We know in the neural network if we want the global minimum, we need the loss function to be convex, so is there any paper shows that and talks about that?
Asked
Active
Viewed 118 times
1 Answers
1
It's not only for Neural Networks, for any learning task, a convex loss function ensures that there is a unique minimum. For most common loss functions, their convexity is fairly straight forward to prove using one of the maths definition of convexity. For example, these slides from the University of Maryland address all this and present convex loss functions along with their curves: http://users.umiacs.umd.edu/~abhishek/cmsc726slides.pdf
Loïc L.
- 46
- 4
-
yes I know all these but in my presentation to my teacher, he said I have to make a reference to a paper that says these and I have searched everywhere and I couldn't find anything – A.omrani Feb 05 '19 at 14:04
-
A quick search on Google Scholar gives you quite a few papers addressing the topic, for example: https://www.mitpressjournals.org/doi/abs/10.1162/089976604773135104 or https://ieeexplore.ieee.org/abstract/document/705577 – Loïc L. Feb 05 '19 at 15:30