How do I convert a ByteString to an appropriately sized Word?

Question

Basically I've read in 5 bytes that correspond to a quantity, but I would like to convert it to a Word64. What's the best way to do this?

Edit: I should also say that this is run in an inner loop so performance is critical. Ideally I'd like to do something like:

uint64_t word = 0;
char bytes[5] = getbytes(5)
word += (bytes[0] << 32) 
        + (bytes[1] << 24) 
        + (bytes[2] << 16) 
        + (bytes[3] << 8) 
        + (bytes[4])

Or something similar.

It's 5 bytes, of what I'm assuming a big-endian word made up of 40 bits — nimish, Sep 20 '11 at 19:27

hammar · Accepted Answer · 2011-09-20T19:51:28.037

1

If you just want a simple function, assuming big-endian byte order, this one should do the job:

foo :: B.ByteString -> Word64
foo = B.foldl' (\x y -> x * 256 + fromIntegral y) 0

However, if you are reading lots of binary data, you might want to look into using the binary package.

edited Sep 20 '11 at 19:51

answered Sep 20 '11 at 17:19

hammar

138,522
17
304
385

Of course, I didn't even think of inserting dummy padding. But that also seems a bit excessive, creating and destroying a few intermediate data structures. – nimish Sep 20 '11 at 19:29
@nt. I wouldn't worry too much about it, but I've added a simpler alternative to my answer. – hammar Sep 20 '11 at 19:46

score 0 · Answer 2 · answered Sep 20 '11 at 22:09

As Hammar said, binary (and the strict bytestring version, cereal) is great, but they are measuably slower than the fastest solutions (they perform the same shifting proposed in Hammar's solution).

I've found that a simple FFI routine is the fastest solution:

getNthWord n b = inlinePerformIO (unsafeUseAsCString b (flip peekElemOff n . castPtr))

If you're willing to add a build-dep, an equally fast solution is to use Vector.Storable from the vector package.

Note neither of these handle your 5 byte big endian format, so you'd need to alter the data producer for this to be useful.

PS The FFI solution assumes word alignment. I originally had a bug when users ran that routine on non-x86 systems.

How do I convert a ByteString to an appropriately sized Word?

2 Answers2