How to keep unreachable code?

Question

I'd like to write a function that would have some optional code to be executed or not depending on user settings. The function is cpu-intensive and having ifs in it would be slow since the branch predictor is not that good.

My idea is making a copy in memory of the function and replace NOPs with a jump when I don't want to execute some code. My working example goes like this:

int Test()
{
    int x = 2;
    for (int i=0 ; i<10 ; i++)
    {
        x *= 2;

        __asm {NOP}; // to skip it replace this
        __asm {NOP}; // by JMP 2 (after the goto)
            x *= 2; // Op to skip or not

        x *= 2;
    }
    return x;
}

In my test's main, I copy this function into a newly allocated executable memory and replace the NOPs by a JMP 2 so that the following x *= 2 is not executed. JMP 2 is really "skip the next 2 bytes".

The problem is that I would have to change the JMP operand every time I edit the code to be skipped and change its size.

An alternative that would fix this problem would be:

__asm {NOP}; // to skip it replace this
__asm {NOP}; // by JMP 2 (after the goto)
goto dont_do_it;
    x *= 2; // Op to skip or not
dont_do_it:
x *= 2;

I would then want to skip or not the goto, which has a fixed size. Unfortunately, in full optimization mode, the goto and the x*=2 are removed because they are unreachable at compilation time.

Hence the need to keep that dead code.

I'm using VStudio 2008.

Answer 1

You can cut the cost of the branch by up to 10, just by moving it out of the loop:

int Test()
{
    int x = 2;
    if (should_skip) {
        for (int i=0 ; i<10 ; i++)
        {
            x *= 2;
            x *= 2;
        }
    } else {
        for (int i=0 ; i<10 ; i++)
        {
            x *= 2;
            x *= 2;
            x *= 2;
        }
    }

    return x;
}

In this case, and others like it, that might also provoke the compiler into doing a better job of optimising the loop body, since it will consider the two possibilities separately rather than trying to optimise conditional code, and it won't optimise anything away as dead.

If this results in too much duplicated code to be maintainable, use a template that takes x by reference:

    int x = 2;
    if (should_skip) {
        doLoop<true>(x);
    } else {
        doLoop<false>(x);
    }

And check that the compiler inlines it.

Obviously this increases code size a bit, which will occasionally be a concern. Whichever way you do it though, if this change doesn't produce a measurable performance improvement then I'd guess that yours won't either.

Answer 2

If the number of permutations for the code is reasonable, you can define your code as C++ templates and generate all variants.

Answer 3

You do not specify what compiler and platform you are using, which will prevent most people from being able to help you. For example, on some platforms, the code section is not going to be writeable, so you won't be able to replace the NOPs with a JMP.

You are trying to pick-and-choose the optimizations offered to you by the compiler and second-guessing it. In general, it's a bad idea. Either write the whole inner loop block in assembly, which would prevent the compiler eliminating is as dead code, or put the damn if statement in there and let the compiler do its thing.

I'm also dubious that the branch prediction is bad enough where you will gain any sort of a net win from doing what you're proposing. Are you sure this isn't a case of premature optimization? Have you written the code in the most obvious way possible and only then determined that its performance isn't good enough? That would be my suggested start.

Answer 4

Here's an actual answer to the actual question!

volatile int y = 0;

int Test() 
{
    int x = 2; 
    for (int i=0 ; i<10 ; i++) 
    { 
        x *= 2; 

        __asm {NOP}; // to skip it replace this 
        __asm {NOP}; // by JMP 2 (after the goto) 
        goto dont_do_it;
    keep_my_code:
        x *= 2; // Op to skip or not 
    dont_do_it: 
        x *= 2; 
    }
    if (y) goto keep_my_code;
    return x; 
} 

Answer 5

Is this x64? You might be able to use function pointers and a conditional move to avoid the branch predictor. Load the address of the procedure based on the user settings; one of the procedures could be a dummy that does nothing. You should be able to do this without any inline ASM at all.

Answer 6

This may give insight:

#pragma optimize for Visual Studio.

That said, for this particular problem I would hand-code into ASM, using the VS asm output as a reference point.

At the meta level, I would have to be very certain this was the best design & algorithm for what I was doing before I started optimizing for the CPU pipe.

Answer 7

If you get this to work then I would profile it to make sure that it really is faster for you. On modern CPUs there is very little you can do that is slower than modifying code that is already in the cpu cache, or worse, the cpu pipeline. The cpu basically has to throw out all the work that is in the pipeline and start again.