3

I'm trying to find global maximum of a Python function with a lot of parameters (500+). Unfortunately I'm unable to make a derivative of this function - what the function does is basically it loops several times over np.array of shape ~ (150000,50) and then does some calculations with the data.

So far I was using scipy.optimize.minimize, method=Powell which seemed to give the best results out of the scipy.optimize.minimize methods.

At first I thought the output of the minimize function was final and the best result that could be found - but then I found out that in some cases - when I save the coefficients and run the minimize function again with the coefficients from the previous run as starting values it gives higher values than the previous run. So what I'm basically doing is the following:

import numpy as np
from scipy.optimize import minimize
from numba import jit

@jit (nopython=True)
def MyFunction (coefs, data):
    # do some calculations
    return value*-1
    
data = np.load('myData.npy', allow_pickle=True)
coefs = np.random.sample(500)   
res = minimize(MyFunction, coefs, args = (data), method='Powell', options={'maxiter': 100000000, 'disp': True, 'return_all': True})

for a in range (0, 10000):
    res = minimize(MyFunction, coefs, args = (data), method='Powell', options={'maxiter': 100000000, 'disp': True, 'return_all': True})
    coefs = res.x
  1. Is there some more effective way?
  2. How can I speed up the code? Yes I already made the code much faster with using jit. But what about threading? Or some other idea?
pepazdepa
  • 117
  • 8
  • 1
    Finding the proven global optimum of a function with 500 decision variables but no gradients is a daunting task. Powell is a local solver, so it will not deliver global solutions. – Erwin Kalvelagen Aug 24 '22 at 17:00
  • Is there a way you can leverage threading inside `MyFunction` itself? – remram Aug 24 '22 at 17:03
  • Well, to what extent this problem is solvable depends pretty strongly on the function involved. I think it would help a lot to revisit the function which, you said, you have code for, and look closer at what's going on there. Post the function if you can. It might seem like too much trouble to extract a mathematical function from that, but that is not much of a price to pay for changing the problem from "probably tremendous amounts of time required" to "somewhat difficult". Also, take a look at automatic differentiation, which, iirc, constructs a gradient for a function expressed only as code. – Robert Dodier Aug 24 '22 at 23:24
  • @RobertDodier Thank you very much for this advice, I didn't know that something like automatic differentiation exists. It looks like libraries such as JAX can compute the derivative of any Python function relatively easily. (In my case, I need to learn how to modify the function, because JAX uses a slightly different syntax, but that's for another discussion.) – pepazdepa Aug 25 '22 at 13:28

0 Answers0