0

I have used caliper based template https://github.com/dcsobral/scala-foreach-benchmark happily for years. It runs randomly constructed problems multiple times and than calculates average time consumption.

Now I have faced with non-deterministic algorithm. So I need to know both running time and resulting fitness. I'm searching for a java/scala benchmark framework that can measure both characteristics in average and worst case scenario.

Non-deterministic means that algorithm relies on some random generator to make decisions. It used to find a near-optimal solution, where searching the optimum would require too much processor time. e.g. solution for the TSP problem.

Fitness means cost function for optimization process. Different runs (with different random seeds) may come with different costs. So you need to stabilize not only run time, but cost value (fitness).

I don't know if repeated invocation of a function until it shows acceptable run time variation is common feature of benchmark frameworks, but the caliper does so, and I search for a similar framework that is more advanced and capable of handling fitness besides time.

ayvango
  • 5,867
  • 3
  • 34
  • 73

1 Answers1

0

Use JMH: it has Scala bindings, and can sample execution time.

Aleksey Shipilev
  • 18,599
  • 2
  • 67
  • 86
  • It helps with one of the problem dimensions: time. But what about the second: fitness (distance to the optimum)? – ayvango Mar 29 '14 at 00:32
  • I don't get what *exactly* do you mean for fitness. Is "optimum" just the best running time ever observed? If so, what's the problem of measuring a few executions of the benchmark and process the result data? – Aleksey Shipilev Mar 30 '14 at 04:50
  • explains precisely searched behavior – ayvango Apr 09 '14 at 15:28
  • @ayvango, sorry, I don't get any sense from these four seemingly random words :) – Aleksey Shipilev Apr 09 '14 at 17:51
  • I'm the one who should apologize. I'd like you to notice recent update to the topic, in which I've added details I suppose sufficient to make precise description of what I seek. – ayvango Apr 09 '14 at 19:08