1

The example app for benchmark.js defines its performance tests as strings rather than JavaScript functions:

https://github.com/bestiejs/benchmark.js/blob/master/example/jsperf/index.html#L250

Is there an advantage to defining the performance test in this way? Why isn't it just a function?

monsur
  • 45,581
  • 16
  • 101
  • 95

2 Answers2

2

From what I can tell, through a long series of calls the function code there ends up at a function in benchmark.js named getSource, which converts it to a string anyway:

/**
 * Gets the source code of a function.
 *
 * @private
 * @param {Function} fn The function.
 * @param {String} altSource A string used when a function's source code is unretrievable.
 * @returns {String} The function's source code.
 */
function getSource(fn, altSource) {
  var result = altSource;
  if (isStringable(fn)) {
    result = String(fn);
  } else if (support.decompilation) {
    // escape the `{` for Firefox 1
    result = (/^[^{]+\{([\s\S]*)}\s*$/.exec(fn) || 0)[1];
  }
  // trim string
  result = (result || '').replace(/^\s+|\s+$/g, '');

  // detect strings containing only the "use strict" directive
  return /^(?:\/\*+[\w|\W]*?\*\/|\/\/.*?[\n\r\u2028\u2029]|\s)*(["'])use strict\1;?$/.test(result)
    ? ''
    : result;
}

This is done within the function "clock", which offers another explanation down the line as to its motiviations to making a benchmark function out of a string each time clock is called:

// Compile in setup/teardown functions and the test loop.
// Create a new compiled test, instead of using the cached `bench.compiled`,
// to avoid potential engine optimizations enabled over the life of the test.

I am no expert of benchmark.js, but my reading of it is that the library deliberately wants to start the benchmark from raw code, forcing the compiler/interpreter to run on code it's never seen before. This does make sense to me, as a hypothetical browser that freezes for five seconds on load to optimize the code might have amazing performance, but it would be unfair to ignore the five seconds it spent compiling in the benchmark. Also, it does a fair bit of modification to the original code in order to control the scope and exception handling; benchmark just needs the code as a string in order to properly manipulate it.

In this view, it isn't so much that passing the functions as strings is particularly important, but there's no reason not to - it's going to end up as a string anyway.

Bubbles
  • 3,795
  • 1
  • 24
  • 25
  • Thanks for the detailed answer. You are right that both the function and the string are eventually turned into a string. However I wasn't sure if there was an extra cost associated with calling code inside a function, vs calling "raw" code outside of a function (which is what you get with a string). – monsur Nov 08 '12 at 18:40
  • Well, you can always call benchmark.js on benchmark.js to get a complete answer ;). My best guess without performing this myself is that there's a very slight performance advantage, since creating a new function will require more initial work by the javascript compiler/interpreter than just a string - I doubt it's really worth worrying about though. – Bubbles Nov 08 '12 at 21:20
  • For the code in example/jsperf/index.html though, some of it would throw an error if it wasn't in a string - the "Error" code he adds isn't even wrapped in a function, and several of the other adds have similar issues (comments outside of function calls and whatnot). I would imagine this would be the biggest motivation to pass them as strings; it allows more robust code formatting than having to shoehorn everything directly into a function. – Bubbles Nov 08 '12 at 21:24
0

Adding the function as a string or reference has no impact on testing the performance in this case.

Look at this benchmark.js#L336 and samples below. Docs say that it's up to you.

You may want to look at a clock function: benchmark.js#L2452. You will see that if a function is passed as a reference it will be converted to string. What's more the whole test is created as a string.

I don't know all advantages why it's a better solution but these come to my mind:

  • it's easier to set up the whole test as one pice of code
  • gives possibility to introduce functions from different sources e.g. database or textarea
  • tests can be saved and re-run

I would like to notice that all advantages I have pointed out have nothing to do with technical aspects of running the code. If it is a string, it must be interpreted and executed defined number of times. Then it's checked how much time has passed and that's the result of a benchmark (basically). The whole concept is the same:

var counter = N,
    after,
    before = performance.now;
while(counter--) {
    myFun();
}
after = performance.now;
console.log(after - before);
dreame4
  • 749
  • 5
  • 9