4

I have been trying to use a Java agent to apply a bytecode transformation with ASM.

I implemented an Agent with the premain method adding a transformer to the Instrumentation. I added the "Premain-Class" line in the .jar manifest

Premain-Class: <MyAgentPath>

Then I tried to run the application with the agent.

There I have a problem : my transformer modifies some method calls, so if not all involved classes are modified too, it cannot work. And there are some classes which are not modified, like "org.apache.commons.math3.util.FastMath". Of course then, I got the error :

java.lang.NoSuchMethodError: org.apache.commons.math3.util.FastMath.floor<new_descriptor>

I checked a lot of posts saying it could be the bootstrap loader which does not know the path to this class so I tried to add it using different ways :

  • Adding the "Boot-Class-Path" line to the manifest :

    Boot-Class-Path: <...>/commons-math3<...>.jar

  • Using the method "appendToBootstrapClassLoaderSearch(JarFile)"

    inst.appendToBootstrapClassLoaderSearch("<...>/commons-math3<...>.jar");

  • Using the JVM argument "-Xbootclasspath/a:"

    -Xbootclasspath/a:<...>/commons-math3<...>.jar

None of this changed anything.

I also used the Instrumentation class method getAllLoadedClasses() to see which ones were loaded, and all classes involved in the application process where loaded, including FastMath.

for(Class<?> clazz : MyAgent.getInstInstance().getAllLoadedClasses()){
    buffWrite.write(clazz.getName());

As the class "FastMath" gave an error and as the Bootstrap Loader should have its path, I tried adding some method calls to methods from other classes in the same package. It appears the problem does not show for every class of the package.

For example: MathUtils is transformed and a call to the modified method checkFinite(D)V -> checkFinite<new_descriptor>.

So I guess the problem has nothing to do with the paths given to the bootstrap loader.

If you have some ideas about what is happening, I would be glad to hear about it!

Rafael Winterhalter
  • 42,759
  • 13
  • 108
  • 192
Jabis
  • 41
  • 6

1 Answers1

2

A NoSuchMethodError is most likely not caused by not adding something to the bootstrap class loader. The only chance that this could be a problem would be if there were suddenly two such jars available where one was instrumented and the other was not.

If you call change a method checkFinite(D)V to become another method checkFinite<new_descriptor>, then you need to make sure that any class using FastMath.floor updates the descriptor to this method. This means that you need to walk through every method of every class looking for visitMethodIns calls of ASM. It seems like you are missing some. Since you are changing the layout of the FastMath class, you must instrument it while loading it for the first time and you cannot redefine it.

The Java internal classes do not know of FastMath as this is a third-party dependency. Therefore, it should be possible to instrument any call from your agent. It seems to me like you are loading FastMath prematurely.

Rafael Winterhalter
  • 42,759
  • 13
  • 108
  • 192
  • Hello Rafael, I am not asking about the `NoSuchMethodError` in this post, this error is caused by the call to `FastMath.floor` but as the FastMath class is not transformed, the method `floor` descriptor does not match. This error is expected if FastMath is not transformed. The transformations works fine, I am already able to transform the entire application via a perturbed ClassLoader. But I would prefer to do it with a Java Agent for convenience purpose. So the problem does not come from ASM usage but from the fact that FastMath is not transformed. – Jabis Dec 15 '15 at 10:51
  • But can you observe that `FastMath` is handed to the transformer (logging, break point). If you refer to the **loaded** class from your agent, it would be loaded prematurely. Otherwise, it should work just fine. – Rafael Winterhalter Dec 15 '15 at 10:57
  • I added breakpoints in the transform() method to see if `FastMath` would appear there. It did not... What I do not understand is why a class like FastMath would be loaded by the agent, but not transformed? – Jabis Dec 15 '15 at 11:12
  • If you load a class **before** the `ClassFileTransformer` is in place, it is not transformed. So I assume that this is happening in your agent. – Rafael Winterhalter Dec 15 '15 at 11:18
  • Ok that sounds logic, but I am using a `premain` method and not `agentmain` so the Agent should be the first "thing" to be loaded by the JVM. Besides, using `getAllLoadedClasses()` method, I can see that `FastMath` is not loaded before a certain point of the code... `for (Class> clazz : MyAgent.getInstInstance().getAllLoadedClasses()){ if(clazz.getSimpleName.equals("FastMath")){ //breakpoint } } ` – Jabis Dec 15 '15 at 11:21
  • Try using `-verbose:class` (http://www.oracle.com/technetwork/java/javase/clopts-139448.html#gbmtm) to see when the class is loaded. – Rafael Winterhalter Dec 15 '15 at 11:33
  • This option shows that `FastMath` is loaded during the transformation of the class which contains the method using `FastMath`. So it is after the `ClassFileTransformer` is set. – Jabis Dec 15 '15 at 12:22
  • Are you using *ASM* with `ClassWriter.COMPUTE_FRAMES`? The automatic computation of frames unfortunately causes class loading of the classes in question. Have a look at: http://asm.ow2.org/asm50/javadoc/user/org/objectweb/asm/ClassWriter.html#getCommonSuperClass%28java.lang.String,%20java.lang.String%29 - If you want to avoid class loading, have a look at Byte Buddy's `TypePool` class and override the `getCommonSuperClass` method using an instance of it: https://github.com/raphw/byte-buddy – Rafael Winterhalter Dec 15 '15 at 12:26
  • Yes, I am using `ClassWriter.COMPUTE_FRAMES`. But as I said, this is not a problem of ASM usage, this is already tested and it works. I will try redefinition of already loaded class. But I think there is something else wrong, I see no reason for not transforming just some classes from `org.apache.commons.math3.util`... – Jabis Dec 15 '15 at 12:39
  • I tried to use the `retransformClasses()` method in the `transform()` method. The call to `isRetransformClassesSupported()` returns true. But this apparently doesn't call the `transform()` method on the class passed in argument... – Jabis Dec 16 '15 at 09:32
  • Have you used the overloaded method for registering a class file transformer? You have to specify explicitly that the transformer is eligable. The second argument, the boolean must be `true`. – Rafael Winterhalter Dec 16 '15 at 09:50
  • Yes I did, when I put a breakpoint on the `retransformClasses()` method, I click on `StepInto` again and again, and the last step is in `InstrumentationImpl.retransformClasses()` and it just incremented the `classRedefinedCount` . But it never went into the `transform()` method I expect it to go in. – Jabis Dec 16 '15 at 10:02
  • Set a break point in your transformer directly. If you registered it for redefinition, I am certain, it is triggered. It probably throws an exception during transformation (those get supressed). – Rafael Winterhalter Dec 16 '15 at 11:52
  • This breakpoint is already set in the transformer, at the line where `retransformedCLasses()` is called. I thought about an exception too, and that is what happened when I forgot (at the first try) to set `Can-Retransform-Classes` to true in the manifest, but now, following the execution trace with the debugger, I do not see any exception created... – Jabis Dec 16 '15 at 12:14
  • ClassFileTransformers supress exceptions silently. – Rafael Winterhalter Dec 16 '15 at 12:17
  • Then this is part of my problem :p because I can't know what is happening... using `-verbose:class` option print this `[Loaded org.apache.commons.math3.util.FastMath from __VM_RedefineClasses__]` FastMath is redefined but not as expected... – Jabis Dec 16 '15 at 12:44
  • If you want to avoid the complexity, you may want consider my library Byte Buddy that could serve you as for convenience: https://github.com/raphw/byte-buddy - Have a look at the `AgentBuilder`. From the `Transformer`, you can apply an ASM `ClassVisitor`. It also adds some convenience for logging and the like. – Rafael Winterhalter Dec 16 '15 at 13:05
  • Thank you for sharing, but I would prefer not to use it, what I need is very simple, and a simple agent with a simple transformer loads every class as expected, but as soon as I add my big transformer, it does not even pass all classes in the `transform` method... My transformer works fine, I am sure of it, but maybe the JVM doesn't like the way I transform the bytecode and skips some classes... this is really confusing... – Jabis Dec 16 '15 at 14:11
  • That is why sometimes it is easier to use a specialized library that tested for all the corner cases. You can always try it for debugging purposes and see what is working out differently. I still argue that you load something prematurely. – Rafael Winterhalter Dec 16 '15 at 14:37
  • The documentation says the retransformation must not change the signature of methods (one thing my transformation does) (see at https://docs.oracle.com/javase/7/docs/api/java/lang/instrument/Instrumentation.html#retransformClasses(java.lang.Class...)) I still don't know, but maybe it is the same for first transformation. At least I know I cannot use `retransformClasses()`` – Jabis Dec 16 '15 at 14:42
  • Ah, of course. With chaning the signature, you are provocing a failure during the instrumentation and the class is not retransformed. The exception is supressed. You must make sure that you never refer to the loaded class, always only use its name! – Rafael Winterhalter Dec 16 '15 at 15:03