L-BFGS from RISO not working

Question

I am testing the implementation of RISO's L-BFGS library for function minimization for logistic regression in Java. Here is the link to the class that I am using.

To test the library, I am trying to minimize the function:

f(x) = 2*(x1^2) + 4*x2 + 5

The library needs the objective and the gradient functions which I implemented as below:

   /**
      The value of the objective function, given variable assignments
      x. This is specific to your problem, so you must override it.
      Remember that LBFGS only minimizes, so lower is better.
   **/
   public double objectiveFunction(double[] x) throws Exception {
        return (2*x[0]*x[0] + 3*x[1] + 1);
   }

   /**
      The gradient of the objective function, given variable assignments
      x.  This is specific to your problem, so you must override it.
   **/
   public double[] evaluateGradient(double[] x) throws Exception {
        double[] result = new double[x.length];
        result[0] = 4 * x[0];
        result[1] = 3;
        return result;
   }

Running the code with this implementation of the objective function and gradient gives me the following exception:

Exception in thread "main" Line search failed. See documentation of routine mcsrch. 
Error return of line search: info = 3 Possible causes: 
function or gradient are incorrect, or incorrect tolerances. (iflag == -1)

I haven't changed the tolerances from the default values. What am I doing wrong?

lennon310 · Answer 1 · 2014-01-10T13:46:41.497

1

I don't think your cost function has a minimum since x2 can reach -Inf, and gradient algorithm won't find it.

It is a quadratic function for x1, but not for x2. I suspect the exception is thrown out because the gradient algorithm cannot find out the optimal solution, and it 'thinks' the problem is the tolerance coefficient is not correctly set, or the gradient function is wrong

Do you mean f(x) = 2*(x^2) + 3*x + 1 in your object function?

edited Jan 10 '14 at 13:46

answered Jan 10 '14 at 01:20

lennon310

12,503
11
43
61

It can because this is a regular quadratic function and quadratic functions always have global optima. – Darth.Vader Jan 10 '14 at 05:42
1

it is a quadratic function for x1, but not for x2. I suspect the exception is thrown out because the gradient algorithm cannot find out the optimal solution, and it 'thinks' the problem is the tolerance coefficient is not correctly set, or the gradient function is wrong – lennon310 Jan 10 '14 at 13:46

L-BFGS from RISO not working

1 Answers1