0

I am working on a non-convex optimization these days and the question comes to my mind about the application of non-convex optimization in deep learning. How can be sure that our objective function is convex? Thanks

Amin
  • 63
  • 10

1 Answers1

1

The standard definition is if f(θx + (1 − θ)y) ≤ θf(x) + (1 − θ)f(y) for 0≤θ≤1 and the domain of x,y is also convex.

So if you could prove that for your function, you would know it's convex.

In deep learning its very difficult to be sure that your objective function is Non Convex thats why initialization and hyperparameter tuning becomes very important

Ashwiniku918
  • 281
  • 2
  • 7
  • Right, we always can take a look at the behaviour of the optimizer such as cost function versus iteration and also play with hyperparameters, but can we be sure that the final solution is the best? – Amin Jan 10 '22 at 17:58