0

I'm writing this program where I have to do a bunch of optimizations. Some with only 1 variable, some with 2. At first I was using the basinhopping algorithm from the scipy.optimize library, but I figured that the normal minimize algorithm should do the job. The basinhopping optimization was working, more or less, but it was extremely time-consuming. Now, I'm using the normal minimize optimization and I've already figured out how to do it for 1 variable. The code for this is given below. I'm using the COBYLA method here, since this one seems to be the only one working. (Nelder-Mead and Powell also work, but sometimes they give back a negative x, which I can't have. And since both of these methods are unconstrained, I can't use them). Hence my first question: what is the difference between all the methods and why do some of the methods converge for my function and others don't?

x0 = [50]
func = lambda x: calculate_score(Sector(center, x, rot), im, msk)
ret = op.minimize(func, x0, method='COBYLA')
print(ret)

The code that I use for the optimization for 2 variables is quite identical to the one for 1 variable, but somehow it gives me the wrong results. Does this have to do with the method I'm using? Or what could be the problem here?

x0 = [50, 50]
func = lambda x: calculate_score(Triangle(center, x[0], x[1], rot), im, msk)
ret = op.minimize(func, x0, method='COBYLA')
print(ret.x[0], ret.x[1])

For the sake of completeness, below is my code for the calculate_score function. I was maybe thinking to calculate the gradient of this function, so that given this gradient the BFGS or L-BFGS-B methods would work, but I'm not quite sure how to do this.

def calculate_score(sect, im, msk):
# find the circle in the image
center, radius = find_circle(im)
# variable to count the score
score = 0
# Loop over all pixels of the detected circle
# This is more time efficient than looping over all pixels
for i in range(0 - radius, radius):
    for j in range(0 - radius, radius):
        pixel = Point(center.x + i, center.y + j)
        # Check if pixel is in given sector
        if sect.in_sector(pixel):
            # Check if pixel is white
            if msk[pixel.y, pixel.x]:
                score -= 1  # Decrement score
            else:
                score += 1  # Increment score
print(score)
return score  # Return score as result

In short, what I would like to know is:

  • Was it a good idea to switch from basinhopping to minimize? (I just thought basinhopping was extremely slow)
  • Is the method COBYLA I'm using the best one for this specific case?
  • Why is my result for 1 variable correct, while my result for 2 variables isn't?
Frederik Vanclooster
  • 443
  • 1
  • 7
  • 21
  • Optimization is a lot of try and error. If you objective function is well behaved and you have some constraints on your variables COBYLA is fine. Basinhopping is able to leave local minima, which makes it a good choice for a bumpy objective function. – Joe Aug 14 '19 at 16:49
  • How can I find out if my objective function if well behaved or not? – Frederik Vanclooster Aug 14 '19 at 17:25
  • Lots of ways, e.g. take a look at how Basinhopping converges. If it jumps out (and it does only if the parameters are set carefully, see the documentation) see where it does jump to. Or start COBYLA or Nelder-Mead using random initial points and see if the converge to the same minimum. Check how they converge visually, plot the iteration steps. There is a lot of try and error involved, use e.g. scipy's `minimize()`, there you can change the optimization algorithm easily. – Joe Aug 14 '19 at 18:42
  • So you mean I have to plot and/or analyse the intermediate values for x, before it jumps out? – Frederik Vanclooster Aug 14 '19 at 18:59
  • Yes. You can get an idea of the convergence behaviour. To choose the algorithm, or e.g. the hopping parameters, size of the dips, their distance, etc. – Joe Aug 16 '19 at 05:07

0 Answers0