0

Suppose, I an algorithm, whose runtime depends on two parameters. I want to find the best set of parameters that minimizes the runtime. The two parameters are continuous double values in the range of 0 to INFINITY.

Therefore, for two parameters a,b: I want to find the best values of a and b that minimize the runtime. I think this is pretty standard practice, but I could not find good literature on this. I found few literature such as MLE, Least Squares, etc. but they talk about distribution.

max
  • 1,692
  • 5
  • 28
  • 40

1 Answers1

0

First use your brains to understand the possible functional relationship between those parameters and the running time, in a qualitative way. This means having a first idea on the number and positions of possible maxima, smoothness of the function, asymptotic behavior and any other clue that you can find.

Then make up your mind about a reasonable range of values where it makes sense to sample the function values. If those ranges are very wide, it is preferable to sample using a geometric progression rather than arithmetic (say, powers of 2).

Then measure and observe the function values with a graphical viewer and confirm your intuitions. It is likely that this will be enough to spot the gross location of the absolute maximum. Finding an accurate position might be useless if it gives you the last percents of improvement. It is also very likely that the location of the optimum will depend on the particular dataset, making accurate location even less useful.

  • I did the following according to your advice. First guessed the value of P1 and P2, and set decided 8 distict values for P1 and P2. Say, P1=(2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9) P2=(0.95 1.05 1.15 1.25 1.35 1.45 1.55 1.65). Then I ran 64 experiments for all combination of P1 and P2, and finding out which gives me the minimum runtime. Next Iteration, I can reduce the step size. Is there a name for this method? – max Mar 20 '16 at 19:58
  • @max: what you did so far is exhaustive search. If you focus the search and reduce the step, you get a variant of the dichotomic search, also related to the Hooke-Jeeves pattern approach. But what matters most is to get insight in the function behavior. Your initial intervals look very tight, I assume you have your reasons. –  Mar 20 '16 at 20:07
  • Thanks, yes I pinpointed this values by doing several iterations of larger ranges. I have to mention this approach in a paper, that is why I need the name of this method. – max Mar 20 '16 at 20:14
  • 1
    As this is done informally, you can just call it "trial and error". –  Mar 20 '16 at 20:15