0

I am trying to do a simple Java bytecode obfuscator which works by replacing GOTO instructions with simple conditional jumps, say, if 10 != 15 GOTO else throw IllegalStateException. My current code is:

    final AbstractInsnNode[] insns = method.instructions.toArray().clone();

    for (final AbstractInsnNode insn : insns) {
        final int op = insn.getOpcode();

        if ((op == GOTO) || (op == IFLE) || (op == IFGE)) {
            LabelNode l0 = new LabelNode();
            LabelNode l1 = new LabelNode();
            LabelNode l2 = new LabelNode();

            int locals = (method.localVariables == null) ? 0 : method.localVariables.size();
            int params = (method.parameters == null) ? 0 : method.parameters.size();

            int v0index = locals + params;
            int v1index = v0index + 1;
            int exindex = v1index + 1;

            // Init fake conditional fields
            method.instructions.insertBefore(insn, new LdcInsnNode(10F));
            method.instructions.insertBefore(insn, new VarInsnNode(FSTORE, v0index));

            method.instructions.insertBefore(insn, new LdcInsnNode(45F));
            method.instructions.insertBefore(insn, new VarInsnNode(FSTORE, v1index));

            // Crossing jumps
            method.instructions.insertBefore(insn, l1);
            method.instructions.insert(insn, l0);
            method.instructions.insert(l0, l2);

            LabelNode l3 = new LabelNode();
            LabelNode l4 = new LabelNode();

            method.instructions.insert(l2, l3);
            method.instructions.insert(l3, l4);

            // If 'v0!=v1', jump to l0, otherwise goto l3
            method.instructions.insertBefore(l1, new VarInsnNode(FLOAD, v0index));
            method.instructions.insertBefore(l1, new VarInsnNode(FLOAD, v1index));
            method.instructions.insertBefore(l1, new InsnNode(FCMPG));
            method.instructions.insertBefore(l1, new JumpInsnNode(IFNE, l0));
            method.instructions.insertBefore(l1, new JumpInsnNode(GOTO, l3));

            // Jump to l3 results in throwing an exception
            // Create and throw the exception
            method.instructions.insertBefore(l4, new TypeInsnNode(NEW, "java/lang/IllegalStateException"));
            method.instructions.insertBefore(l4, new InsnNode(DUP));
            method.instructions.insertBefore(l4, new MethodInsnNode(INVOKESPECIAL, "java/lang/IllegalStateException", "<init>", "()V", false));
            method.instructions.insertBefore(l4, new InsnNode(ATHROW));

            method.instructions.insertBefore(l0, new JumpInsnNode(GOTO, l2));
            method.instructions.insertBefore(l2, new JumpInsnNode(GOTO, l1));

            // Exception handler
            LabelNode start = new LabelNode();
            LabelNode handler = new LabelNode();
            LabelNode end = new LabelNode();

            method.instructions.insertBefore(l0, start);

            method.instructions.insert(l2, end);
            method.instructions.insert(end, handler);

            // Just throw the exception again
            LabelNode l5 = new LabelNode();

            method.instructions.insert(handler, l5);
            method.instructions.insertBefore(l5, new TypeInsnNode(NEW, "java/lang/IllegalStateException"));
            method.instructions.insertBefore(l5, new InsnNode(DUP));
            method.instructions.insertBefore(l5, new MethodInsnNode(INVOKESPECIAL, "java/lang/IllegalStateException", "<init>", "()V", false));
            method.instructions.insertBefore(l5, new InsnNode(ATHROW));

            // Try/catch
            TryCatchBlockNode tryBlock = new TryCatchBlockNode(start, end, handler, "java/lang/IllegalStateException");
            method.tryCatchBlocks.add(tryBlock);

            // Init local variables
            method.visitLocalVariable("_v0_" + Rand.alphaNumeric(5), "F", null, l0.getLabel(), l2.getLabel(), v0index);
            method.visitLocalVariable("_v1_" + Rand.alphaNumeric(5), "F", null, l0.getLabel(), l2.getLabel(), v1index);
            method.visitLocalVariable("_ex_" + Rand.alphaNumeric(5), "Ljava/lang/IllegalArgumentException;", null, start.getLabel(), handler.getLabel(), exindex);
        }
    }

Where method is the obfuscation method's parameter of type MethodNode, and the class implements interface Opcodes.

This works fine, but not with all methods (I'm quite new to bytecode, and so don't know the exact cases). For example, it works fine for the main method:

Original Java code (decompiled in Procyon): https://p.reflex.rip/DLMT.cs

Original bytecode: https://p.reflex.rip/ywJt.go

Obfuscated Java code (decompiled in Procyon): https://p.reflex.rip/Er9V.cs

Obfuscated bytecode: https://p.reflex.rip/JBAb.go

However, it breaks one of the others, divMinByMax, method:

Original Java code (decompiled in Procyon): https://p.reflex.rip/AW9W.java

Original bytecode: https://p.reflex.rip/GX2k.cpp

Obfuscated Java code (decompiled in Procyon, FAILED): https://p.reflex.rip/Eqju.java

Obfuscated bytecode: https://p.reflex.rip/isiX.cpp

This method causes a VerifyError when I try to run the obfuscated JAR with java -jar:

Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.VerifyError: Inconsistent stackmap frames at branch target 27
Exception Details:
  Location:
    test/one/HelloRandom.divMinByMax(DD)D @21: goto
  Reason:
    Current frame's stack size doesn't match stackmap.
  Current Frame:
    bci: @21
    flags: { }
    locals: { double, double_2nd, float, float }
    stack: { }
  Stackmap Frame:
    bci: @27
    flags: { }
    locals: { double, double_2nd, float, float }
    stack: { 'java/lang/IllegalStateException' }
  Bytecode:
    0x0000000: 2826 9712 7145 1272 4624 2596 9a00 0ca7
    0x0000010: 0014 9e00 48a7 0006 a7ff fabb 0016 59b7
    0x0000020: 0073 bfbb 0016 59b7 0073 bf00 0000 0000
    0x0000030: 00bf 0000 00bf 0000 0000 0000 00bf 0000
    0x0000040: bf00 00bf 0000 bf00 00bf 0000 0000 0000
    0x0000050: 00bf 0000 0000 0000 00bf 2826 6faf
  Exception Handler Table:
    bci [24, 27] => handler: 27
  Stackmap Table:
    full_frame(@18,{Double,Float,Float},{Integer})
    same_locals_1_stack_item_frame(@24,Integer)
    same_locals_1_stack_item_frame(@27,Object[#22])
    same_locals_1_stack_item_frame(@35,Integer)
    full_frame(@43,{},{Object[#159]})
    same_locals_1_stack_item_frame(@50,Object[#159])
    same_locals_1_stack_item_frame(@54,Object[#159])
    same_locals_1_stack_item_frame(@62,Object[#159])
    same_locals_1_stack_item_frame(@65,Object[#159])
    same_locals_1_stack_item_frame(@68,Object[#159])
    same_locals_1_stack_item_frame(@71,Object[#159])
    same_locals_1_stack_item_frame(@74,Object[#159])
    same_locals_1_stack_item_frame(@82,Object[#159])
    append_frame(@90,Double,Float,Float)
    same_locals_1_stack_item_frame(@93,Double)

I did a lot of research, and the only thing I found was the cause: as I understood, the problem is that the stack on @21 (GOTO which jumps to the label which does throw new IllegalStateException):

stack: { }

(which is empty) does not match the stack at @27, on the jump target label:

stack: { 'java/lang/IllegalStateException' } (which contains the exception it should "throw").

So basically, the error, as I understand, happens when I'm trying to perform a GOTO <n> jump, where <n> is the number of the label which "throws" an IllegalStateException.

How can I fix this issue? Maybe there is a way to make the stack at @21 contain java/lang/IllegalStateException before jumping too (so that these two stacks, the one before and the one after jump, match)? Or something else I can do with it?

German Vekhorev
  • 339
  • 6
  • 16

1 Answers1

2

You are inserting an exception handler after the instruction, but when the instruction is a conditional branch, i.e. IFLE or IFGE, the branch might not be taken and the code flow continues after the instruction, running into the exception handler.

This creates an inconsistent state as the exception handler expects a Throwable on the stack which doesn’t exist when the code flow continues after the instrumented instruction. But of course, you do not want to execute the exception handler in that case, so you have to insert another GOTO, from l2 to l5 if I got it right.

This is not a problem when instrumenting GOTO instructions, which never proceed after the instruction.

At this place, I’d recommend a different coding style. Inserting before and after different reference nodes makes it impossible to predict the actual code structure when reading your code. It would be much more maintainable, if you just inserted a linear list of instructions using only one reference node.

Holger
  • 285,553
  • 42
  • 434
  • 765