"Mechanically generated" java source files in the Java source code

Question

As I was looking through the Java source code, I found some unusual files, mostly related to ByteBuffers in the java.nio package which had a very messy source code and were labelled This file was mechanically generated: Do not edit!.

These files also contained large portions of blank lines (some even in the middle of javadocs (!!?)), presumably to prevent the line numbers from changing. I have also seen a few java decompilers, such as procyon-decompiler, which have an option to keep line numbers, but I doubt that's the case, because putting blank lines before the final accolade changes nothing.

Here are a few of these files (I couldn't find any links to them online and didn't pastebin them because I don't want to break any copyright, but you can find them in the src.zip folder at the root of your JDK installation folder):

java.nio.ByteBuffer
java.nio.DirectByteBufferR
java.nio.Bits
java.nio.BufferOverflowException

I'd be curious to know:

Which tool generated these files?
Why does the tool keep the line numbers the same? Is it to make debugging (stacktraces) easier?
Why would a tool be used to generate them, while all other classes are programmed by humans?
Why would the tool put blank lines randomly inside parentheses, before the final accolade, or even in javadocs?

I doubt you'll get an answer as that code appears to have been there a long time- check out [this blog post from 2006](http://www.iggdawg.com/blog/2006/09/jaaaavaaaaa/) , when Java was still owned by Sun. — APC, Jul 23 '14 at 07:28
It seems that these files are generated during the build by some preprocessor from template files: http://hg.openjdk.java.net/jdk9/dev/jdk/file/3b298c230549/src/share/classes/java/nio/ByteBufferAs-X-Buffer.java.template — Andreas Fester, Jul 23 '14 at 07:33
IIRC, the C preprocessor inserts blank lines due to skips after #if or #else. Here, the rationale is clear, I think: if some compiler flags an error in the ouput, you'll find it in the original input. — laune, Jul 23 '14 at 07:46

Andreas Fester · Accepted Answer · 2014-07-23T08:06:55.560

I can probably not answer all of the questions, but some background is:

In the Makefile at http://hg.openjdk.java.net/jdk7/jdk7/jdk/file/9b8c96f96a0f/make/java/nio/Makefile, they are generating different java source files from the same template file through some preprocessor:

...
$(BUF_GEN)/CharBuffer.java: $(X_BUF_TEMPLATE) $(GEN_BUFFER_SH)
    $(prep-target)
    @$(RM) $@.temp
    TYPE=char SRC=$< DST=$@.temp $(GEN_BUFFER_CMD)
    $(MV) $@.temp $@
$(BUF_GEN)/ShortBuffer.java: $(X_BUF_TEMPLATE) $(GEN_BUFFER_SH)
    $(prep-target)
    @$(RM) $@.temp
    TYPE=short SRC=$< DST=$@.temp $(GEN_BUFFER_CMD)
    $(MV) $@.temp $@
...

$(X_BUF_TEMPLATE) refers to X-Buffer.java.template, which is the source for typed buffers like CharBuffer, ShortBuffer and some more.

Note: The URLs might change in the future. Also sorry for referring to Java 7 - in Java 8 they have modified the build system, I did not find the corresponding Makefiles so far.

Which tool generated these files?

GEN_BUFFER_SH / GEN_BUFFER_CMD finally refers to genBuffer.sh, so the script which creates these files is http://hg.openjdk.java.net/jdk7/jdk7/jdk/file/9b8c96f96a0f/make/java/nio/genBuffer.sh.

Why would a tool be used to generate them, while all other classes are programmed by humans?

I don't have an authoritative answer for this specific case, but usually you are using code generation tools

if you need to create a lot of similar classes/methods which only differ in some detail, but which is subtle enough so that you can not use established mechanisms like generics or method parameters (probably the case here, since the buffers are generated for primitive types which can not be used with Generics)
if you need to create complex algorithms from a much simpler representation (like generating parsers from a grammar).

Why does the tool keep the line numbers the same? Is it to make debugging (stacktraces) easier?

I am guessing: yes, its to retain the line numbers in stack traces so that they match the template files. Other tools like the C preprocessor work similar.

Yes these classes are diversified for primitive types (short, char, ...). A rare case. There is some investigation to have primitive types wrapped in some future java, so you could have `Buffer`. — Joop Eggen, Jul 23 '14 at 08:03

"Mechanically generated" java source files in the Java source code

1 Answers1