Automatic parameter tuning

Question

I've got a an audio processing app that takes an input audio file, processes it, and spits out a modified output audio file. This audio processing app has 10-15 parameters that affect how it processes the audio, and thus affects the content of the output audio file (it might have, say, a different frequency response, be louder, quieter, etc.). All these parameters have constrained ranges (x0 must be < 1 and > -1 for example).

The output audio file is evaluated by a tool that gives it a score. This tool knows what the "ideal" output should sound like, and scores the output file accordingly. A score of 1.0 means the output is ideal, i.e. the input file was processed with the best possible parameter set. A score of 0 means the output is completely wrong.

So with 10-15 parameters with their valid ranges, the combinations are endless! I'd be sitting here manually tweaking these parameters forever until I got the best solution. I've checked out some LP/MIP solvers (CBC, MS Solver Foundation, GKLP) but these use a mathematical equation as an objective function... you don't "plug in" an external evaluation function as far as I can see.

Is a LP/MIP solver the right tool to aid in the parameter tuning? Any ideas?

Thanks,

akevan

score 1 · Accepted Answer · answered Aug 05 '11 at 17:33

1

You could use a general heuristic like simulated annealing or genetic algorihms. Your evaluation process would be the fitness/objective function.

answered Aug 05 '11 at 17:33

Paweł Obrok

22,568
8
74
70

I'm now using simulated annealing... specifically from chapter 10 of Numerical Recipes in C. It is definitely the right approach although I have to figure out the best way to add constraints to the algorithm. – akevan Aug 09 '11 at 17:55

score 0 · Answer 2 · answered Nov 29 '12 at 20:22

You could use the SPOT packet (R programming language). It allows you to find (near-)optimal parameter settings using significantly less runs than brute force. You can use any programming language for your fitness function code, SPOT has an adapter for that, and offers an automatic mode with default setup (You don't have to worry about the design types and prediction models). It has a steep learning curve, but once you understood the basics, it is a powerful tool. Here is a quick guide; chapter 2.6 offers a concrete example. The SPOT package comes with several examples.

score 0 · Answer 3 · edited Nov 26 '13 at 17:30

If you had the objective function, then yes LP would be the ideal approach (and would give the ideal answer); the solution would be purely analytic. But in the absence of the function it seems you've correctly understood the problem becomes an integer programming problem. I have less knowledge of integer programming, but I believe that too assumes an objective function to solve. Even with the function, integer programs are NP-hard.

So it seems you would need to use brute force to detect a local maxima, and then tune it. I realize that is exactly what you didn't want to do, but that is what comes to mind.

Automatic parameter tuning

3 Answers3