UInt64 gets iffy when doing big math?

Question

I have a strange result coming from some big math & have no clue as to why I'm getting a different answer from vb.net vs python.

Here are the quick snippets & results:

VB.NET
Dim MSB As UInt32 = 3067297518
Dim LSB As UInt32 = 1439785590
Dim sqln As UInt64 = MSB * (2 ^ 32) + LSB 

Python:
sqln = msb * (2 ** 32) + lsb

Python Result: 13173942528351756918
VB RESULT:     13173942528351756288

Note: I also tries declaring sqln as a ULong and a Double (same answer) The MSB and LSB are a match in both debuggers - !! ??

Any Ideas? =+ My thanks

Outstanding Jon - very eloquent & it works! One little follow up could you suggest a fix for the final piece? I believe the same sort of thing is happening even though you got my sqln straightened out :)

python says: = bdntyxtax2smq
vb.net says: = bfpuzytbx3s00

VB.NET
Dim sqlid As String = ""
Dim alphabet As String = "0123456789abcdfghjkmnpqrstuvwxyz"
For iCount = 0 To 12
    sqlid = alphabet((sqln / (32 ^ iCount)) Mod 32) + sqlid
Next

Python:
for i in range(0, 13):
    sqlid = alphabet[(sqln / (32 ** i)) % 32] + sqlid

score 4 · Accepted Answer · answered Mar 01 '12 at 20:40

I've decompiled your VB code as C#, and it looks like this:

uint MSB = 0xb6d33eee;
uint LSB = 0x55d16276;
ulong sqln = (ulong) Math.Round((double) ((num2 * 4294967296) + num));

This is because the power operator in VB always returns a Double:

The result is number raised to the power of exponent, always as a Double value.

I would suggest using the shift operator, and making all your variables UInt64, so that the shifting is done to a 64-bit integer:

Dim MSB As UInt64 = 3067297518
Dim LSB As UInt64 = 1439785590
Dim sqln As UInt64 = (MSB << 32) + LSB

That gives the right answer. (You don't actually need LSB to be a UInt64, but you might as well do everything with 64-bit integers.)

UInt64 gets iffy when doing big math?

1 Answers1