How to easily create Java bytecode dependent regression tests?

Question

I have identified a bug in my application (which processes bytecode using soot) that only arises on specific bytecode instructions.

I want to create a test for that specific case. However, I'm not able to reliably write test code, that will compile to the expected bytecode, which would then trigger the bug.

This is my attempt to trigger the bug:

public void updateRhsOnIfEq() {
        int x = 15;
        int y = AircraftControl.readSensor(0);
        // FIXME != in bytecode instead of ==
        if (x == y) {
            AircraftControl.readSensor(y);
        }
        else {
            AircraftControl.readSensor(x);
        }
    }

The problem is, that the compiler changes the branch logic by inverting the comparison and switching the two branches. As you can see in the bytecode below, it does a != comparison instead of ==. However, the bug I'm testing for is only triggered by a ==.

 public void updateRhsOnIfEq();
     0  bipush 15
     2  istore_1 [x]
     3  iconst_0
     4  invokestatic AircraftControl.readSensor(int) : int [17]
     7  istore_2 [y]
     8  iload_1 [x]
     9  iload_2 [y]
    10  if_icmpne 21 <============================== Should be if_icmpeq
    13  iload_2 [y]
    14  invokestatic AircraftControl.readSensor(int) : int [17]
    17  pop
    18  goto 26
    21  iload_1 [x]
    22  invokestatic AircraftControl.readSensor(int) : int [17]
    25  pop
    26  return

Is there a way to write test cases that need to result in predictable bytecode easily? Is this possible at all given that there are different Java compilers, versions thereof etc?

johnchen902 · Answer 1 · 2013-05-16T12:32:13.367

The compiler doesn't change the branch logic. ~~It's the compiler's natural behavior to use if_icmpne in this case (just my opinion)~~ To make (eclipse) compiler use if_icmpeq, just change your code as following:

if (x != y) {
    AircraftControl.readSensor(x);
}
else {
    AircraftControl.readSensor(y);
}

This code:

public static void main(String[] args) {
    int x = (int) System.currentTimeMillis(), y = (int) System
            .currentTimeMillis();
    if (x != y) {
        System.out.println("x != y");
    } else {
        System.out.println("x == y");
    }
}

Result in:

   0: invokestatic  #16     // Method java/lang/System.currentTimeMillis:()J
   3: l2i
   4: istore_1
   5: invokestatic  #16     // Method java/lang/System.currentTimeMillis:()J
   8: l2i
   9: istore_2
  10: iload_1
  11: iload_2
  12: if_icmpeq     26
  15: getstatic     #22     // Field java/lang/System.out:Ljava/io/PrintStream;
  18: ldc           #26     // String x != y
  20: invokevirtual #28     // Method java/io/PrintStream.println:(Ljava/lang/String;)V
  23: goto          34
  26: getstatic     #22     // Field java/lang/System.out:Ljava/io/PrintStream;
  29: ldc           #34     // String x == y
  31: invokevirtual #28     // Method java/io/PrintStream.println:(Ljava/lang/String;)V
  34: return

This is interesting, I could swear that I tried that and it didn't work. Perhaps that was a different case. Can you elaborate on the `compiler's natural behavior`? Is this documented, can I rely on that, why does it do that? — phant0m, May 15 '13 at 14:54
It’s neither specified nor guaranteed. But reversing the conditional is indeed natural, [as explained here](https://stackoverflow.com/a/42563601/2711488)… — Holger, Jan 22 '18 at 08:17

Antimony · Answer 2 · 2013-05-16T03:46:35.233

If you want specific bytecode instructions, the obvious and most reliable approach is to write it directly in bytecode.

I've written an open source assembler available here. For simple cases, you can get by with something like Jasmin, which will probably be better documented. I also have a disassembler, so if you only need minor tweaks, you can just compile a Java class, disassemble, make the tweak and then reassemble.

score 1 · Answer 3 · answered May 15 '13 at 14:53

Addressing the question of how to generate test cases, you are correct. Different Java compilers and different versions of the same compiler are likely to produce different bytecodes ... over time.

Your alternatives are:

if the compiler can generate the the testcases for you, then use it
or you could:
- take an existing compiler generated testcase that is similar
- disassemble it to make sure you know where the bytes are in the file
- use a hex editor to tweak the instructions; e.g. change the if_icmpne to if_icmpeq
or you could write the example code from scratch using a Java assembly language.

Once you've generated the test cases you need to capture and store them as classfiles. Don't rely on being able to regenerate them on the fly.

How to easily create Java bytecode dependent regression tests?

3 Answers3