1

I am using JDK 15. (I am using ByteBuddy 1.10.16 to generate some classes but it's mostly irrelevant here, I think, except as background information.)

In one of these generated classes, I am calling invokeExact() on a MethodHandle constant I've managed to store in the generated class. It is a "field setter" acquired via MethodHandles.Lookup#findSetter.

(In what follows I am aware of the MethodHandles.privateLookupIn() method.)

I've noticed that the "field setter" MethodHandle in question fails when it represents a private field. At most levels this does not surprise me: a direct MethodHandle is, well, direct: while I don't pretend to know much about the innards of all this stuff, it seems to me that surely it must just wrap some low-level bytecode devoid of access checks.

But given the existence of privateLookupIn() which shows that bypassing access checks is possible in certain situations, is there a path where I can "harvest" a "field setter" MethodHandle from class A that can read a private field, and then store it as a constant in another class B such that invokeExact() on it will succeed?

I believe I have done something similar in the past (have to check) involving private methods, but in those cases I was not using MethodHandle constants, i.e. I was acquiring the MethodHandle at class initialization time during <clinit> time using privateLookupIn() and storing the resulting MethodHandle in a private static final field, and then calling invokeExact() on the contents of that field. If I have to continue to go this route, I will, but MethodHandle constants seem appealing here and it would be nice to use them if I can.

So another way of phrasing my question is: is the constant form in which a MethodHandle is represented capable of storing its privileges? Or is there some one-time way of "upping" the privileges given a MethodHandle stored as a constant? Or does the fact that a given MethodHandle is stored as a constant prevent it for all time from accessing anything other than conventionally accessible Java constructs? I didn't see anything super obvious in the JVM specification in the relevant section.

Laird Nelson
  • 15,321
  • 19
  • 73
  • 127
  • 2
    I don't know about the specifics of ByteBuddy, but in general the access checks are performed when the constant is resolved. You can get around this by pre-resolving the MethodHandle and then using constant pool patching to put the live object in the constant pool, but there is currently no public API available that does that (although I believe at was/is planned: https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/java/lang/invoke/MethodHandles.java#L315). – Jorn Vernee Oct 11 '20 at 01:41
  • 2
    @JornVernee Afaik, there are no plans to bring constant pool patching into a public API Since [`defineHiddenClass`](https://docs.oracle.com/en/java/javase/15/docs/api/java.base/java/lang/invoke/MethodHandles.Lookup.html#defineHiddenClass(byte%5B%5D,boolean,java.lang.invoke.MethodHandles.Lookup.ClassOption...)) is supposed to supersede the old `Unsafe.defineAnonymousClass`, the constant pool patching is about to disappear without replacement. You can use dynamic constants, to load such a method handle via `ldc` instructions or such alike. Though, there’s no benefit over a`static final` field. – Holger Oct 17 '20 at 17:40
  • 2
    @Holger note that there is also a [defineHiddenClassWithClassData](https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/java/lang/invoke/MethodHandles.java#L2003) method, that is currently package private, which could be used as a replacement for constant pool patching. (Though you're right, they are not exactly the same). [Last I heard](https://bugs.openjdk.java.net/browse/JDK-8220607?focusedCommentId=14305026&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14305026) this was the plan to replace CP patching, but I haven't asked recently... – Jorn Vernee Oct 17 '20 at 23:32

1 Answers1

3

The specification you’ve linked states:

To resolve MH, all symbolic references to classes, interfaces, fields, and methods in MH's bytecode behavior are resolved, using the following four steps:

R is resolved. This occurs as if by field resolution (§5.4.3.2) when MH's bytecode behavior is kind 1, 2, 3, or 4, and as if by method resolution (§5.4.3.3) when MH's bytecode behavior is kind 5, 6, 7, or 8, and as if by interface method resolution (§5.4.3.4) when MH's bytecode behavior is kind 9.

The linked chapters, i.e. §5.4.3.2 for fields, describe the ordinary resolution process, including access control. Even without that explicit statement, you could derive the existence of access control from the preceding description, that states that these symbolic method handle references are supposed to be equivalent to specific listed bytecode behavior.

So a direct method handle acquired via a CONSTANT_MethodHandle_info entry of the class file’s constant pool can not access classes or members that wouldn’t be also accessible directly by bytecode instructions.

But since JDK 11, you can use Dynamic Constants to load constants of arbitrary type defined by an arbitrary bootstrapping process. So when you can express how to get the constant in terms of Java code, like the use of privateLookupIn, you can also define it as bootstrapping of a dynamic constant and load that constant at places where you would otherwise load the direct method handle.

Consider the following starting point:

public class DynConstant {
    private static void inacessibleMethod() {
        new Exception("inacessibleMethod() called").printStackTrace();
    }

    public static void main(String[] args) throws Throwable {
        // express the constant
        Handle theHandle = new Handle(H_INVOKESTATIC,
            Type.getInternalName(DynConstant.class), "inacessibleMethod",
            Type.getMethodDescriptor(Type.VOID_TYPE), false);

        String generatedClassName
                = DynConstant.class.getPackageName().replace('.', '/')+"/Test";

        ClassWriter cw = new ClassWriter(0);
        cw.visit(55, ACC_INTERFACE|ACC_ABSTRACT,
                generatedClassName, null, "java/lang/Object", null);

        MethodVisitor mv = cw.visitMethod(
                ACC_PUBLIC|ACC_STATIC, "test", "()V", null, null);
        mv.visitCode();
        mv.visitLdcInsn(theHandle);
        mv.visitMethodInsn(INVOKEVIRTUAL,
                "java/lang/invoke/MethodHandle", "invokeExact", "()V", false);
        mv.visitInsn(RETURN);
        mv.visitMaxs(1, 0);
        mv.visitEnd();
        cw.visitEnd();
        byte[] code = cw.toByteArray();

        ToolProvider.findFirst("javap").ifPresentOrElse(javap -> {
            String fName = generatedClassName+".class";
            try {
                Path dir = Files.createTempDirectory("javapTmp");
                Path classFile = dir.resolve(fName);
                Files.createDirectories(classFile.getParent());
                Files.write(classFile, code);
                javap.run(System.out, System.err, "-c", "-cp",
                    dir.toAbsolutePath().toString(), generatedClassName);
                for(Path p = classFile;;p=p.getParent()) {
                    Files.delete(p);
                    if(p.equals(dir)) break;
                }
            } catch (IOException ex) {
                throw new UncheckedIOException(ex);
            }
        }, () -> System.out.println("javap not found in current environment"));

        try {
            MethodHandles.Lookup lookup = MethodHandles.lookup();
            lookup.findStatic(lookup.defineClass(code),
                "test", MethodType.methodType(void.class)).invokeExact();
        }
        catch(Throwable t) {
            t.printStackTrace();
        }
    }
}

It tries to define a new runtime class that attempts to load a MethodHandle constant pointing to inacessibleMethod() via a CONSTANT_MethodHandle_info. The program prints

interface instexamples.Test {
  public static void test();
    Code:
       0: ldc           #12                 // MethodHandle REF_invokeStatic instexamples/DynConstant.inacessibleMethod:()V
       2: invokevirtual #17                 // Method java/lang/invoke/MethodHandle.invokeExact:()V
       5: return
}
java.lang.IllegalAccessError: class instexamples.Test tried to access private method 'void instexamples.DynConstant.inacessibleMethod()' (instexamples.Test and instexamples.DynConstant are in unnamed module of loader 'app')
    at instexamples.Test.test(Unknown Source)
    at instexamples.DynConstant.main(DynConstant.java:100)

Now, let’s change the constant to a dynamic constant that will perform the equivalent to

MethodHandles.Lookup l = MethodHandles.lookup();
l = MethodHandles.privateLookupIn(DynConstant.class, l);
MethodHandle mh = l.findStatic(
        DynConstant.class, "inacessibleMethod", MethodType.methodType(void.class));

when the constant is resolved the first time. The definition of the constant is “a bit” more involved. Since the code contains three method invocations, the definition requires three method handles, further, another handle to the already existing bootstrap method ConstantBootstraps.invoke(…) that allows to use arbitrary method invocations for the bootstrapping. These handles can be used to define dynamic constants, whereas dynamic constants are allowed as constant input to another dynamic constant.

So we replace the definition after the // express the constant comment with:

Type string = Type.getType(String.class), clazz = Type.getType(Class.class);
Type oArray = Type.getType(Object[].class), object = oArray.getElementType();
Type mhLookup = Type.getType(MethodHandles.Lookup.class);
Type mHandle = Type.getType(MethodHandle.class), mType = Type.getType(MethodType.class);
Type targetType = Type.getType(DynConstant.class);

String methodHandles = Type.getInternalName(MethodHandles.class);

Handle methodHandlesLookup = new Handle(H_INVOKESTATIC, methodHandles,
    "lookup", Type.getMethodDescriptor(mhLookup), false);
Handle privateLookupIn = new Handle(H_INVOKESTATIC, methodHandles,
    "privateLookupIn", Type.getMethodDescriptor(mhLookup, clazz, mhLookup), false);
Handle findStatic = new Handle(H_INVOKEVIRTUAL, mhLookup.getInternalName(),
    "findStatic", Type.getMethodDescriptor(mHandle, clazz, string, mType), false);
Handle invoke = new Handle(H_INVOKESTATIC,
    Type.getInternalName(ConstantBootstraps.class), "invoke",
    Type.getMethodDescriptor(object, mhLookup, string, clazz, mHandle, oArray), false);

ConstantDynamic methodHandlesLookupC = new ConstantDynamic("lookup",
    mhLookup.getDescriptor(), invoke, methodHandlesLookup);
ConstantDynamic privateLookupInC = new ConstantDynamic("privateLookupIn",
    mhLookup.getDescriptor(), invoke, privateLookupIn, targetType, methodHandlesLookupC);
ConstantDynamic theHandle = new ConstantDynamic("findStatic",
    mHandle.getDescriptor(), invoke, findStatic,
    privateLookupInC, targetType, "inacessibleMethod", Type.getMethodType("()V"));

To avoid repeating the very long constant method descriptor strings, I use ASM’s Type abstraction. In principle, we could use constant strings for all type names and signatures.

This program prints:

interface instexamples.Test {
  public static void test();
    Code:
       0: ldc           #45                 // Dynamic #2:findStatic:Ljava/lang/invoke/MethodHandle;
       2: invokevirtual #50                 // Method java/lang/invoke/MethodHandle.invokeExact:()V
       5: return
}
java.lang.Exception: inacessibleMethod() called
    at instexamples.DynConstant.inacessibleMethod(DynConstant.java:23)
    at instexamples.Test.test(Unknown Source)
    at instexamples.DynConstant.main(DynConstant.java:89)

The complexity of a dynamic constant composed of three constants created by method invocations will result in quite a big constant pool. We may generate a custom bootstrap method instead and get a significantly smaller class file, despite we have an additional method:

public class DynConstant {
    private static void inacessibleMethod() {
        new Exception("inacessibleMethod() called").printStackTrace();
    }

    public static void main(String[] args) throws Throwable {
        Type string = Type.getType(String.class), clazz = Type.getType(Class.class);
        Type mhLookup = Type.getType(MethodHandles.Lookup.class);
        Type mHandle = Type.getType(MethodHandle.class), mType = Type.getType(MethodType.class);

        Type targetType = Type.getType(DynConstant.class);

        String myBootstrapName = "privateLookup";
        String myBootstrapDesc = Type.getMethodDescriptor(mHandle, mhLookup, string, clazz, clazz, mType);

        String generatedClassName = DynConstant.class.getPackageName().replace('.', '/')+"/Test";

        Handle myBootStrap = new Handle(H_INVOKESTATIC, generatedClassName,
            myBootstrapName, myBootstrapDesc, true);
        ConstantDynamic theHandle = new ConstantDynamic("inacessibleMethod",
            mHandle.getDescriptor(), myBootStrap, targetType, Type.getMethodType("()V"));

        ClassWriter cw = new ClassWriter(0);
        cw.visit(55, ACC_INTERFACE|ACC_ABSTRACT, generatedClassName, null, "java/lang/Object", null);

        MethodVisitor mv = cw.visitMethod(ACC_PUBLIC|ACC_STATIC, "test", "()V", null, null);
        mv.visitCode();
        mv.visitLdcInsn(theHandle);
        mv.visitMethodInsn(INVOKEVIRTUAL, "java/lang/invoke/MethodHandle", "invokeExact", "()V", false);
        mv.visitInsn(RETURN);
        mv.visitMaxs(1, 0);
        mv.visitEnd();
        mv = cw.visitMethod(ACC_PRIVATE|ACC_STATIC, myBootstrapName, myBootstrapDesc, null, null);
        mv.visitCode();
        mv.visitVarInsn(ALOAD, 3); // bootstrap argument, i.e. DynConstant.class
        mv.visitVarInsn(ALOAD, 0); // MethodHandles.lookup() generated as JVM arg
        mv.visitMethodInsn(INVOKESTATIC, "java/lang/invoke/MethodHandles", "privateLookupIn",
            Type.getMethodDescriptor(mhLookup, clazz, mhLookup), false);
        mv.visitVarInsn(ALOAD, 3); // bootstrap argument, i.e. DynConstant.class
        mv.visitVarInsn(ALOAD, 1); // invoked name, i.e. "inacessibleMethod"
        mv.visitVarInsn(ALOAD, 4); // bootstrap argument, i.e. MethodType ()V
        mv.visitMethodInsn(INVOKEVIRTUAL, "java/lang/invoke/MethodHandles$Lookup", "findStatic",
            Type.getMethodDescriptor(mHandle, clazz, string, mType), false);
        mv.visitInsn(ARETURN);
        mv.visitMaxs(4, 5);
        mv.visitEnd();
        cw.visitEnd();
        byte[] code = cw.toByteArray();

        ToolProvider.findFirst("javap").ifPresentOrElse(javap -> {
            String fName = generatedClassName+".class";
            try {
                Path dir = Files.createTempDirectory("javapTmp");
                Path classFile = dir.resolve(fName);
                Files.createDirectories(classFile.getParent());
                Files.write(classFile, code);
                javap.run(System.out, System.err, "-p", "-c", "-cp",
                    dir.toAbsolutePath().toString(), generatedClassName);
                for(Path p = classFile;;p=p.getParent()) {
                    Files.delete(p);
                    if(p.equals(dir)) break;
                }
            } catch (IOException ex) {
                throw new UncheckedIOException(ex);
            }
        }, () -> System.out.println("javap not found in current environment"));

        try {
            MethodHandles.Lookup lookup = MethodHandles.lookup();
            lookup.findStatic(lookup.defineClass(code),
                "test", MethodType.methodType(void.class)).invokeExact();
        }
        catch(Throwable t) {
            t.printStackTrace();
        }
    }
}
interface instexamples.custombootstrap.Test {
  public static void test();
    Code:
       0: ldc           #18                 // Dynamic #0:inacessibleMethod:Ljava/lang/invoke/MethodHandle;
       2: invokevirtual #23                 // Method java/lang/invoke/MethodHandle.invokeExact:()V
       5: return

  private static java.lang.invoke.MethodHandle privateLookup(java.lang.invoke.MethodHandles$Lookup, java.lang.String, java.lang.Class, java.lang.Class, java.lang.invoke.MethodType);
    Code:
       0: aload_3
       1: aload_0
       2: invokestatic  #29                 // Method java/lang/invoke/MethodHandles.privateLookupIn:(Ljava/lang/Class;Ljava/lang/invoke/MethodHandles$Lookup;)Ljava/lang/invoke/MethodHandles$Lookup;
       5: aload_3
       6: aload_1
       7: aload         4
       9: invokevirtual #35                 // Method java/lang/invoke/MethodHandles$Lookup.findStatic:(Ljava/lang/Class;Ljava/lang/String;Ljava/lang/invoke/MethodType;)Ljava/lang/invoke/MethodHandle;
      12: areturn
}
java.lang.Exception: inacessibleMethod() called
    at instexamples.custombootstrap.DynConstant.inacessibleMethod(DynConstant.java:22)
    at instexamples.custombootstrap.Test.test(Unknown Source)
    at instexamples.custombootstrap.DynConstant.main(DynConstant.java:91)

The bootstrap method has been designed to be reusable. It receives all necessary information as constant arguments, so different ldc instructions can use it to get handles to different members. The JVM does already pass the caller’s lookup context as first argument, so we can use this and don’t need to call MethodHandles.lookup(). The class to search for the member is the first additional argument, which is used as first argument to both, privateLookupIn and findStatic. Since every dynamic constant has a standard name argument, we can use it to denote the member’s name. The last argument denotes the MethodType for the method to look up. When we retrofit this for field lookups, we could remove that parameter, as the third standard argument, the expected constant type could be matched with the expected field’s type.

Basically, the custom bootstrap method does the privateLookupIn based lookup you described in your question, but using it with ldc allows to have lazy initialization (rather than the class initialization time of static final fields) while still getting optimized like static final fields once the instruction has been linked. Also, these dynamic constants are permitted as constant input to other bootstrap methods for other dynamic constants or invokedynamic instructions (though, you can also adapt an existing static final field to a dynamic constant using this bootstrap method).

Holger
  • 285,553
  • 42
  • 434
  • 765
  • Thanks for this; this is independently the exact approach I ended up going with some time ago. It is very powerful indeed. – Laird Nelson Nov 05 '20 at 18:05
  • In your example, I would simply put the bootstrap method into `DynConstant` - makes it more readable. – Johannes Kuhn Nov 05 '20 at 22:51
  • 1
    @JohannesKuhn it depends on the actual use case whether relying on an already existing bootstrap method is feasible. For the example, it was important to have the bootstrap method not in the same class as the target method, to demonstrate getting a handle to an inaccessible method. There are other scenarios imaginable where the declaring class contains a bootstrap method allowing certain `friend` classes to access them that way… – Holger Nov 06 '20 at 08:38
  • There is seldom a need to write the actual bootstrap class in bytecode. Yes, the bootstrap class could use it's own lookup in your example if you move it to DynConstant. But that's not the point - the point is: you probably have an easier time to write the bootstrap method in plain java. – Johannes Kuhn Nov 06 '20 at 09:59
  • 1
    I’m not trying to predict what will be more likely to be used in the future. I tried to give a complete picture. Since the answer explains the principles of the bootstrap method and also provides the plain java equivalent, there should be sufficient information, to allow readers to write their own bootstrap method using whichever approach they prefer. – Holger Nov 06 '20 at 11:49