15

Java 8 appears to generate classes to represent lambda expressions. For instance, the code:

  Runnable r = app::doStuff;

Manifests, roughly, as:

  // $FF: synthetic class
  final class App$$Lambda$1 implements Runnable {
    private final App arg$1;

    private App$$Lambda$1(App var1) {
        this.arg$1 = var1;
    }

    private static Runnable get$Lambda(App var0) {
        return new App$$Lambda$1(var0);
    }

    public void run() {
        this.arg$1.doStuff();
    }
  }

As I understand this, the code is generated at runtime. Now, suppose one wanted to inject code into the run method of the above class. Experiments thus far yield a mix of NoClassDefFound and VerifyError:

java.lang.NoClassDefFoundError: App$$Lambda$2
    at App$$Lambda$2/1329552164.run(Unknown Source)
    at App.main(App.java:9)
Caused by: java.lang.ClassNotFoundException: App$$Lambda$2
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 2 more

This is running against:

$ java -version
java version "1.8.0_51"
Java(TM) SE Runtime Environment (build 1.8.0_51-b16)
Java HotSpot(TM) 64-Bit Server VM (build 25.51-b03, mixed mode)

This is even before pushing any new bytecode into the class.

Is this expected? Smells like a JDK bug, but I'm happy to be wrong!

Here's a Github repo illustrating the behavior

Holger
  • 285,553
  • 42
  • 434
  • 765
BitPusher
  • 990
  • 1
  • 9
  • 20
  • Same behavior holds for the latest flavor of the JDK, `1.8.0_65` – BitPusher Dec 08 '15 at 17:27
  • 1
    The verifiy error indicates that you created broken byte code. Did you try to debug the code? During what action does the error occur? – Rafael Winterhalter Dec 09 '15 at 10:38
  • This is on a retransform call. The ClassTransformation callback is not even invoked in this case! To be precise, this occurs on `Instrumentation#retransformClasses(...)` – BitPusher Dec 09 '15 at 16:06
  • 1
    I tested a lambda retransformation and it works without a problem. You must have a problem in your generated code! – Rafael Winterhalter Dec 09 '15 at 16:30
  • Here's an example of the type of issue I'm encountering with a simple, home brewed javaagent: https://github.com/akilman/java-lambda-retransform. Granted, this generates a distinct error of `NoClassDefFoundError` and `ClassNotFoundException` - this is part of the pattern I'm observing. Roughly, lambda instances simultaneously exist and don't at the same time, depending on the context – BitPusher Dec 10 '15 at 02:24
  • 1
    You return an empty array from your transformer. That cannot go well. Rather return `null` to indicate that you do not want to transform the class. – Rafael Winterhalter Dec 10 '15 at 09:47
  • Thanks much for your input btw. Much appreciated. Updated, and still stuck with the `NoClassDefFoundError`. Which is more or less the theme I'm encountering elsewhere with other transformations and agents. – BitPusher Dec 10 '15 at 16:07
  • 3
    Keep in mind that the mapping of lambda expressions and runtime classes is intentionally unspecified. Multiple lambda expressions may share a class or the same expression may get represented by different, changing classes at runtime. The specification explicitly states these possibilities. Thus, even the Instrumentation API gets fixed to allow you to instrument such a class you are walking on thin ice. What happens to work with one specific version of a particular JVM implementation may fail right in the next revision. – Holger Dec 15 '15 at 11:37
  • 2
    Whatever you want to achieve, you better do it by instrumenting either, the creating `invokedynamic` instruction or the target method. There shouldn’t be any reason to instrument the ephemeral anonymous classes of a lambda expression or method reference. – Holger Dec 15 '15 at 11:38
  • I agree. Instrumenting the generated classes ties one to a major java revision with a particular lambda representation. However, major releases of Java are relatively infrequent. Its not an end all be all solution, but works for a decent period of time. – BitPusher Dec 16 '15 at 05:05
  • On a different note, it appears this behavior is specific to instance method references. Anonymous method bodies and static methods appear un-impacted. Updated the noted [Github repo](https://github.com/akilman/java-lambda-retransform) with cases exercising the same. – BitPusher Dec 16 '15 at 05:07
  • 2
    You can’t assume that this behavior keeps the same until the next major update. Since it’s explicitly unspecified, it may change right the next minor revision. It wouldn’t be the first time for such a change of internals. E.g., the internal string representation changed fundamentally in `7u6`, and the string de-duplication feature was added in `8u20`… – Holger Dec 18 '15 at 15:01

2 Answers2

12

To me, this seems like a bug in the JVM. The system class loader attempts to locate the transformed class by its name. However, lambda expressions are loaded via anonymous class loading where the following condition:

clazz.getClassLoader()
     .loadClass(clazz.getName().substring(0, clazz.getName().indexOf('/')))

yields a ClassNotFoundException resulting in the NoClassDefError. The class is not considered a real class and such anonyoumous classes are for example not passed to a ClassFileTransformer outside of a retransform.

All in all, the instrumentation API feels a bit buggy to me when dealing with anonymous classes. Similarly, LambdaForms are passed to ClassFileTransformers but with all arguments but the classFileBuffer set to null what breaks the transformer class's contract.

For your example, the problem seems to be that you return null; the problem goes away when returning the classFileBuffer what is a no-op. This is however not what the ClassFileTransformer suggests, where returning null is the recommended way of doing this:

a well-formed class file buffer (the result of the transform), or null if no transform is performed.

To me, this seems like a bug in HotSpot. You should report this issue to the OpenJDK.

All in all, it is perfectly possible to instrument anonymously loaded classes as I demonstrate in my code manipulation library Byte Buddy. It requires some unfortunate tweaks compared to normal instrumentation but the runtime supports it. Here is an example that successfully runs as a unit test within the library:

Callable<String> lambda = () -> "foo";

Instrumentation instrumentation = ByteBuddyAgent.install();
ClassReloadingStrategy classReloadingStrategy = ClassReloadingStrategy.of(instrumentation)
    .preregistered(lambda.getClass());
ClassFileLocator classFileLocator = ClassFileLocator.AgentBased.of(instrumentation, 
     lambda.getClass());

assertThat(lambda.call(), is("foo"));

new ByteBuddy()
  .redefine(lambda.getClass(), classFileLocator)
  .method(named("call"))
  .intercept(FixedValue.value("bar"))
  .make()
  .load(lambda.getClass().getClassLoader(), classReloadingStrategy);

assertThat(lambda.call(), is("bar"));
Rafael Winterhalter
  • 42,759
  • 13
  • 108
  • 192
  • Thanks Rafael. I'm reaching out to Oracle and OpenJDK folks. Will update the posting once I get something concrete. – BitPusher Dec 11 '15 at 20:31
5

Bug submission was accepted by folks at Oracle, and is being tracked as JDK-8145964. This isn't exactly a solution, but appears to be a real runtime issue.

BitPusher
  • 990
  • 1
  • 9
  • 20