bitmasking and binary arithmetic

Question

I'd like to know the science behind the following. a 32 bit value is shifted left 32 times in a 64 bit type, then a division is performed. somehow the precision is contained within the last 32 bits and in order to retrieve the value as a floating point number, I can multiply by 1 over the max value of an unsigned 32 bit int.

phase = ((uint64) 44100 << 32) / 48000;
(phase & 0xffffffff) * (1.0f / 4294967296.0f);// == 0.918749988

the same as

(float)44100/48000;// == 0.918749988

**−1** Not the real code. Trying your first example in Python the expression `(phase & 0xffffff) * (1.0 / 4294967296.0)` yields `0.0007812499534338713`. — Cheers and hth. - Alf, Apr 14 '15 at 12:53
For the actual example that you've seen somewhere, note that the maximum value of an unsigned 32-bit int is 2^32-1, very close to the factor of 2^32 that you've introduced. So the results (for that original example) would be nearly identical. — Cheers and hth. - Alf, Apr 14 '15 at 12:58
please remove the downvote, i missed 2 digits out, i have edited. — cool mr croc, Apr 14 '15 at 13:21

Ethouris · Answer 1 · 2015-04-14T13:31:06.810

(...)

If you lose precision when dividing two integer numbers, you should remember the remainder.
The reminder in C++ can be taken by doing 44100%48000 in your case.
Actually these are constants and it's completely clear that 44100/48000 == 0, so remainder is all you have.
Well, the reminder will even be -- guess what -- 44100!
The float type (imposed by the explicit cast) has only 6 significant digits. So 4294967296.0f will be simply 429496e4 (in mathematics: 429496*10^4). That's why this type isn't valuable for anything but playing around.
The best way to get a value of fixed integer type in which all bits are set, and not miss the correct number of 'f' in 0xfffff, is to use the ~ operator and 0 value. In your case, ~uint32_t(0).

Well, I should have said this in the beginning: 44100.0/48000 should give you the result you want. :P

my question isnt why I lose precision, it's what is the process involved in this style of division. — cool mr croc, Apr 14 '15 at 13:23
So what's your question then? Your statement in the "question" is too unclear to guess what you want to know. Does point #5 answer it? — Ethouris, Apr 14 '15 at 13:29

cool mr croc · Accepted Answer · 2015-04-14T16:08:57.267

0

this is the answer I was looking for

bit shifting left will provide that number of bits in which to store the precision vale from a division.

dividing the integer value represented by these bits by 2 to the power of the bit shift amount will return the precision value

e.g

0000 0001 * 2^8 = 1 0000 0000 = 256(base 10)

1 0000 0000 / 2 = 1000 0000 = 128(base 10)

128 / 2^8 = 0.5

edited Apr 14 '15 at 16:08

answered Apr 14 '15 at 15:30

cool mr croc

725
1
13
33

bitmasking and binary arithmetic

2 Answers2