Is gcc reordering local variables at compilation time?

Question

I'm currently reading (for the second time) "Hacking : The Art of Exploitation" and have stumbled on something.

The book suggests two different ways to exploit these two similar programs : auth_overflow and auth_overflow2

In the first one, there is a password checking function layed out like this

int check_authentication(char *password) {
    int auth_flag = 0;
    char password_buffer[16];

    strcpy(password_buffer, password);
    ...
}

Inputing more than 16 ASCII characters will change the value of auth_flag to something greater than 0, thus bypassing the check, as shown on this gdb output:

gdb$ x/12x $esp
0xbffff400: 0xffffffff  0x0000002f  0xb7e0fd24  0x41414141
0xbffff410: 0x41414141  0x41414141  0x41414141  0x00000001
0xbffff420: 0x00000002  0xbffff4f4  0xbffff448  0x08048556

password_buffer @ 0xbffff40c
auth_flag @ 0xbffff41c

The second program inverts the two variables :

int check_authentication(char *password) {
    char password_buffer[16];
    int auth_flag = 0;

    strcpy(password_buffer, password);
    ...
}

The author then suggests than it's not possible to overflow into auth_flag, which I really believed. I then proceeded to overflow the buffer, and to my surprise, it still worked. The auth_flag variable was still sitting after the buffer, as you can see on this gdb output:

gdb$ x/12x $esp
0xbffff400: 0xffffffff  0x0000002f  0xb7e0fd24  0x41414141
0xbffff410: 0x41414141  0x41414141  0x41414141  0x00000001
0xbffff420: 0x00000002  0xbffff4f4  0xbffff448  0x08048556

password_buffer @ 0xbffff40c
auth_flag @ 0xbffff41c

I'm wondering if gcc is not reordering local variables for alignement/optimization purposes.

I tried to compile using -O0 flag, but the result is the same.

Does one of you knows why is this happening ?

Thanks in advance.

"Reordering" somehow implies that you expect there to be an initial "order". C++ does not actually specify any kind of order of storage of local variables, whose storage is called "automatic" -- i.e. don't ask don't tell. — Kerrek SB, Aug 11 '16 at 08:32
I expected the compiler (with its optimizations disabled) to leave the ordering as declared in the code. But I don't know much about compilers. The author of the book seems to expect the same behavior too. I still managed to make it work like I wanted by declaring both variables volatile. — rgehan, Aug 11 '16 at 08:38
Well, your expectations are unfortunately uncorrelated with the language rules. (Sorry, you said C, not C++, but the same point applies.) "Reality different from personal expectations" would be an appropriate summary of your experience :-) Automatic storage is just "automatically there", with very few guaranteed specifics. — Kerrek SB, Aug 11 '16 at 08:40
You're totally right. I find it weird that the first example of the book is based on something inaccurate. I managed to make it work as they expected, but if it's not a general rule then it shouldn't be presented like so — rgehan, Aug 11 '16 at 08:42
There might not even be order: On some system `auth_flag` might not even be in memory, it could be in the CPU register, making the exploit impossible. It seems that author of the book is using a very specific system, and compiler with very specific behaviour. — user694733, Aug 11 '16 at 08:43
That is quite fascinating. Out of curiosity, is there anyway to make the compiler use registers for this. I'd quite like to see the different way the same function can be achieved. Maybe a specific compiler flag ? — rgehan, Aug 11 '16 at 08:51
Well, there is standard `register` keyword, but it's mostly a hint only, so most compilers just ignore it. GCC might have some specific methods, but I am not familiar enough to recommend them. — user694733, Aug 11 '16 at 10:17

chqrlie · Accepted Answer · 2016-10-17T09:43:21.750

The compiler authors are completely free to implement any allocation scheme for local variables with automatic storage. auth_flag could be set before or after password_buffer on the stack, it could be in a register, it could be elided completely if proper analysis of the code allows it. There might not even be a stack... The only guarantee the Standard gives you is this:

strcpy(password_buffer, password); invokes undefined behavior if the source string including its null terminator is longer than the destination array password_buffer. Whether this undefined behavior fits your needs is completely outside of the language specification.

As a matter of fact, some implementors purposely complicate the task of would be hackers by randomizing the behavior in cases such as the posted code.

That's what Kerrek SB and user694733 pointed out. I didn't consider that the output from a compiler could be totally different from the one of another compiler. This totally makes sense. Thanks :) — rgehan, Aug 11 '16 at 09:06

score 0 · Answer 2 · answered May 10 '20 at 10:18

I had the same problem. In order to fix this, put the two variables in a struct. In a struct the fields are always located as defined in the struct. Be aware that the order is reversed.

struct myStruct {
       int auth_flag;
       char password_buffer[16];
};

score 0 · Answer 3 · answered Dec 18 '20 at 21:49

0

I know thats an old question.

But in my case -fno-stack-protector flag did the trick. So if I compile with -fno-stack-protector, local variables ordered as excepted (at least for this simple program ).

I wonder, maybe reordering can be some sort of protection. Here I found link about that

answered Dec 18 '20 at 21:49

Vahag Vardanyan

301
6
13

I used the -fno-stack-protector to prevent the code from detecting the overflow exploit, but it does not affect the ordering of my variables – polmonroig Aug 14 '22 at 12:03

Is gcc reordering local variables at compilation time?

3 Answers3

Linked