How come right shifting by 64-bits is the identity instead of being nilpotent (returning 0)?

Question

This code speaks for itself, but I can't post the question unless I say something here!

#include <stdio.h>
#include <limits.h>

void rightshift_test(int const shift_amount) {
  unsigned long a = 0xabcd1234abcd1234UL;
  printf("a: %lx, shift_amount:%d\n", a, shift_amount);
  unsigned long const b = a >> shift_amount;
  printf("b: %lx\n", b);
}

int main() {
  rightshift_test(56);
  rightshift_test(60);
  rightshift_test(61);
  rightshift_test(62);
  rightshift_test(63);
  rightshift_test(64);
  return 0;
}

And it still won't let me post the question. Here the code is running:

gcc -Wall -Wextra -Werror foo.c -o foo
./foo
a: abcd1234abcd1234, shift_amount:56
b: ab
a: abcd1234abcd1234, shift_amount:60
b: a
a: abcd1234abcd1234, shift_amount:61
b: 5
a: abcd1234abcd1234, shift_amount:62
b: 2
a: abcd1234abcd1234, shift_amount:63
b: 1
a: abcd1234abcd1234, shift_amount:64
b: abcd1234abcd1234

And here are the details of my compiler and my machine, in case that wasn't very obvious.

$ gcc --version
i686-apple-darwin10-gcc-4.2.1 (GCC) 4.2.1 (Apple Inc. build 5666) (dot 3)

$ echo $MACHTYPE
x86_64-apple-darwin10.0

Shifting by more than the data-type size is undefined behavior. End of story. — Mysticial, Jul 06 '12 at 09:58
http://stackoverflow.com/questions/3394259/weird-behavior-of-right-shift-operator — Mysticial, Jul 06 '12 at 10:00
To add to comment by "Mystical" above: complete round shift is probably (incorrectly) optimised as NOP. — Germann Arlington, Jul 06 '12 at 10:05
To add to that comment, it's not an incorrect optimization, because anything is correct once the program invokes UB :-) On some hardware, shift instructions in effect mask the operand with `0x3F` because they only look at the bottom bits. I don't know whether x86-64 does that, but naturally the result is that `>>64` has the same effect as `>>0`. — Steve Jessop, Jul 06 '12 at 11:06
@SteveJessop x86-64 does that, yes. The mask depends on the operand size. — harold, Jul 07 '12 at 12:39
As stated above, Intel shift/rotate operations mask the shift amount with the operand size so a shift of 64 will result in a shift of 0. On ARM, this is not the case and the result of shifting greater than the operand size will output 0. — BitBank, Jul 12 '12 at 16:27
I tried the code on my x86_64 Mac. Confirming the above, a 64-bit shift gave 0xabcd1234abcd1234, 65 bits gave 0x55e6891a55e6891a, and 66 bits gave 0x2af3448d2af3448d. — Steve Summit, Jun 19 '21 at 12:10

paxdiablo · Answer 1 · 2012-09-13T08:23:34.420

The ISO C99 standard has this little snippet dealing with the bitshift operators:

shift-expression:
       additive-expression
       shift-expression << additive-expression
       shift-expression >> additive-expression

The integer promotions are performed on each of the operands. The type of the result is that of the promoted left operand. If the value of the right operand is negative or is greater than or equal to the width of the promoted left operand, the behavior is undefined.

In other words, once you start trying to shift a 64-bit (in your implementation) unsigned long by 64 bits or more, all bets are off. It can return 0, the original value, or 42. It can even format your hard drive and mail porn to your boss (though I've never seen an implementation actually do this).

You may find that some implementations (or even the underlying hardware of the implementation) will optimise these instructions by effectively only using the relevant bits. In other words, for a 64-bit value, it may well only use the least significant seven bits (effectively and-ing it with 0x3f) to give you an operand of 0 through 63.

In that case, since 64 & 0x3f is equal to zero, it's effectively a no-op.

In any case, why it's acting the way it is is supposition. Undefined behaviour is something you really should avoid. If you really want the behaviour to work as you expect, change:

unsigned long const b = a >> shift_amount;

into something like:

unsigned long const b = (shift_amount < 64) ? a >> shift_amount : 0;

as seen in this code (64-bit values are ULLs on my system):

#include <stdio.h>
#include <limits.h>

void rightshift_test(int const shift_amount) {
    unsigned long long a = 0xabcd1234abcd1234ULL;
    printf("a: %llx, shift_amount:%d: ", a, shift_amount);
    unsigned long long const b = (shift_amount < 64) ? a >> shift_amount : 0;
    printf("b: %llx\n", b);
}

int main() {
    rightshift_test(56);
    rightshift_test(60);
    rightshift_test(61);
    rightshift_test(62);
    rightshift_test(63);
    rightshift_test(64);
    rightshift_test(99);
    return 0;
}

The output of this is:

a: abcd1234abcd1234, shift_amount:56: b: ab
a: abcd1234abcd1234, shift_amount:60: b: a
a: abcd1234abcd1234, shift_amount:61: b: 5
a: abcd1234abcd1234, shift_amount:62: b: 2
a: abcd1234abcd1234, shift_amount:63: b: 1
a: abcd1234abcd1234, shift_amount:64: b: 0
a: abcd1234abcd1234, shift_amount:99: b: 0

How come right shifting by 64-bits is the identity instead of being nilpotent (returning 0)?

1 Answers1