1

Consider the following simple optimization problem.

from symfit import parameters, Eq, Ge, Fit, log
from symfit.core.minimizers import BasinHopping

n = 3
# xdata = np.sort(np.random.choice(range(1, 4*n), n))
xdata = [2, 8, 11]
print(xdata)
p1, p2, p3 = parameters('p1, p2, p3')
model = p1*p2*p3
# model = log(p1)+log(p2)+log(p3)
constraints = [
    Eq(xdata[0]*p1+(xdata[1]-xdata[0])*p2+(xdata[2]-xdata[1])*p3, 1),
    Ge(p1, p2),
    Ge(p2, p3),
    Ge(p3, 0.00001)
    ]

fit2 = Fit(- model, constraints=constraints)
print(fit2.execute(options={"ftol": 1e-12}))
fit0 = Fit(- model, constraints=constraints, minimizer=BasinHopping)
print(fit0.execute())

This gives the optimum as: 

Parameter Value        Standard Deviation
p1        1.666668e-01 nan
p2        7.407405e-02 nan
p3        7.407405e-02 nan

for both fit0 and fit2. The total run time for both is about 3 seconds. BasinHopping uses 567 iterations.

Now let's simply take logs of the objective function. So we have:

model = log(p1)+log(p2)+log(p3)

instead of the model = line above. This should give exactly the same result and indeed fit2 works and takes about 1 second. However, the following happens:

fit0 (that is BasinHopping) takes 110137 iterations and 7 minutes to compute the optimum.

What is going wrong here?

Simd
  • 19,447
  • 42
  • 136
  • 271
  • So BasinHopping is significantly slower only when using the log model, but not for the normal model? – tBuLi Apr 12 '19 at 21:01
  • @tBuLi Yes. Also note that slsqp takes fewer than 50 iterations and basinhopping around 110,000 which is quite a spectacular slowdown. – Simd Apr 12 '19 at 21:12
  • Well for one thing, the original model is linear in the parameters, and so it will be much easier to optimize for any minimizer. So in this particular case, I would actually say that the original form is the most natural form for this model. Therefore I'm curious about your reasons for taking the log. Is it an academic question, or does your problem somehow need that? :) – tBuLi Apr 13 '19 at 10:16
  • As to why BasinHopping is slower, in principle BasinHopping will repeatedly execute SLSQP to perform a local minimization from another starting point. If the landscape of the objective after taking the log is for example very flat, then it will converge much more slowly, and it does so many times. That could be (part of) the answer. Perhaps you can try playing around with the options to basinhopping as listed in the scipy docs and feed them to `Fit.execute()` as keyword arguments. – tBuLi Apr 13 '19 at 10:20
  • If you take logs the optimization problem becomes convex which in principle makes it easier to solve. – Simd Apr 13 '19 at 11:06
  • If BasinHopping calls slsqp and slsqp without BasinHopping is very fast, this is confusing – Simd Apr 13 '19 at 11:07
  • 1
    Well, BasinHopping is a global minimizer so it is expected to take longer because by calling it you are implicitly assuming that you expect more than one minimum to exist. But I agree that it is weird that it would take so much longer in this scenario, I have no straightforward answer for that. – tBuLi Apr 13 '19 at 13:30
  • The original model is not linear in the parameters. We are trying the maximize the product of three variables. – Simd Apr 13 '19 at 13:40
  • Why not? If you derive the model w.r.t. any of those parameters, the derivative does not depend on that parameter. Is that not the definition of linearity? – tBuLi Apr 13 '19 at 15:31
  • @tBuLi I may not have the definition right but I think of linear objective functions as in https://en.wikipedia.org/wiki/Linear_programming . That is where the objective function is a linear function of the variables. In this case p1, p2 and p3. p1*p2*p3 is not a linear function of those variables. – Simd Apr 13 '19 at 20:09
  • Yeah you are right that it is not linear in that sense, it can not be written as a system of linear equations. Maybe I as a theoretical physicist also use a slightly different definition than is conventional in numerical optimization. ;) – tBuLi Apr 14 '19 at 16:36

0 Answers0