how to find surrogate of a non convex function

Question

I am doing learning to rank.I have found that some of the loss functions for example 0/1 loss can not be directly minimized being either non-convex or discontinuous etc.this is true in case of other loss functions also.

So the researchers use another type of loss function called "convex surrogate" which "bounds" the loss of the 0/1 type and they try to minimize the surrogate of the loss function to find the parameters(if I am understanding rightly).

my question in this is that what is the procedure to find the surrogate function given a non convex loss function?

where I can read the steps that I have a non convex loss function and I want to device its surrogate loss?

also How I know that some function is an upper bound on the loss of 0/1.

and how to come up with that bound?

Answer 1

Designing of the good surrogate function is a research topic, as defining such function actually leads to the construction of new machine learning model. There is no "rule" to do so - it is what research is about. From practical perspective you rather should explore numerous existing functions, as they are not "just popular" - they are simply good, well understood and usefull.

And how do you check whether a loss function bounds your current one? You provide a mathematical proof. You have true loss l(x,y,p) and a surrogate s(x,y,p) and all you have to do is to show that l(x,y,p)<=s(x,y,p), so you provide a proof of inequality. Again, there is no one rule, this is just applied mathematics, analysis 101.