Why does Closure Compiler insist on adding more bytes?

Question

If I give Closure Compiler something like this:

window.array = '0123456789'.split('');

It "compiles" it to this:

window.array="0,1,2,3,4,5,6,7,8,9".split(",");

Now as you can tell, that's bigger. Is there any reason why Closure Compiler is doing this?

Answer 1

I think this is what's going on, but I am by no means certain...

The code that causes the insertion of commas is tryMinimizeStringArrayLiteral in PeepholeSubstituteAlternateSyntax.java.

That method contains a list of characters that are likely to have a low Huffman encoding, and are therefore preferable to split on than other characters. You can see the result of this if you try something like this:

"a b c d e f g".split(" "); //Uncompiled, split on spaces
"a,b,c,d,e,f,g".split(","); //Compiled, split on commas (same size)

The compiler will replace the character you try to split on with one it thinks is favourable. It does so by iterating over the characters of the string and finding the most favourable splitting character that does not occur within the string:

// These delimiters are chars that appears a lot in the program therefore
// probably have a small Huffman encoding.
NEXT_DELIMITER: for (char delimiter : new char[]{',', ' ', ';', '{', '}'}) {
  for (String cur : strings) {
    if (cur.indexOf(delimiter) != -1) {
      continue NEXT_DELIMITER;
    }
  }
  String template = Joiner.on(delimiter).join(strings);
  //...
}

In the above snippet you can see the array of characters the compiler claims to be optimal to split on. The comma is first (which is why in my space example above, the spaces have been replaced by commas).

I believe the insertion of commas in the case where the string to split on is the empty string may simply be an oversight. There does not appear to be any special treatment of this case, so it's treated like any other split call and each character is joined with the first appropriate character from the array shown in the above snippet.

---

Another example of how the compiler deals with the split method:

"a,;b;c;d;e;f;g".split(";"); //Uncompiled, split on semi-colons
"a, b c d e f g".split(" "); //Compiled, split on spaces

This time, since the original string already contains a comma (and we don't want to split on the comma character), the comma can't be chosen from the array of low-Huffman-encoded characters, so the next best choice is selected (the space).

---

Update

Following some further research into this, it is definitely not a bug. This behaviour is actually by design, and in my opinion it's a very clever little optimisation, when you bear in mind that the Closure compiler tends to favour the speed of the compiled code over size.

Above I mentioned Huffman encoding a couple of times. The Huffman coding algorithm, explained very simply, assigns a weight to each character appearing the the text to be encoded. The weight is based on the frequency with which each character appears. These frequencies are used to build a binary tree, with the most common character at the root. That means the most common characters are quicker to decode, since they are closer to the root of the tree.

And since the Huffman algorithm is a large part of the DEFLATE algorithm used by gzip. So if your web server is configured to use gzip, your users will be benefiting from this clever optimisation.

Answer 2

This issue was fixed on Apr 20, 2012 see revision: https://code.google.com/p/closure-compiler/source/detail?r=1267364f742588a835d78808d0eef8c9f8ba8161

Answer 3

Ironically, split in the compiled code has nothing to do with split in the source. Consider:

Source  : a = ["0","1","2","3","4","5"]
Compiled: a="0,1,2,3,4,5".split(",")

Here, split is just a way to represent long arrays (long enough for sum of all quotes + commas to be longer than split(","") ). So, what's going on in your example? First, the compiler sees a string function applied to a constant and evaluates it right away:

'0123456789'.split('') => ["0","1","2","3","4","5","6","7","8","9"]

At some later point, when generating output, the compiler considers this array to be "long" and writes it in the above "split" form:

["0","1","2","3","4","5","6","7","8","9"] => "0,1,2,3,4,5,6,7,8,9".split(",")

Note that all information about split('') in the source is already lost at this point.

If the source string were shorter, it would be generated in the array array form, without extra splitting:

Source  : a = '0123'.split('')
Compiled: a=["0","1","2","3"]