2

I'm storing numbers in their byte equivalent format, using the least number of bytes possible. With the range 65535 through 16777215, BitConverter gives me a 4 byte array, but I want to only store 3 bytes.

For the code below, my array is [0]254, [1]255, [2]255, [3]0, so I can chop out byte [3]. This is on a Core i7 proc. In my production code, before the array copy, I am checking BitConverter.IsLittleEndian to determine that I can chop the last byte.

int i = 16777214;
byte[] bytesTemp = BitConverter.GetBytes(i);

byte[] value = null;
if (BitConverter.IsLittleEndian) 
    Array.Copy(bytesTemp, 0, value, 0, 3);

My question is - do I need to concern myself with the Endian-ness of the system, or does the CLR just use this LittleEndian format regardless? I don't have a BigEndian system (nor even know how I'd get one) to test whether my byte array comes out in the reverse order.

Christos
  • 53,228
  • 8
  • 76
  • 108
winnt93
  • 43
  • 1
  • 6
  • It depends on the computer architecture, most architectures are LittleEndian. However, .Net code can run on BigEndian architectures - the .NET Micro Framework can run on big-endian. Version 4.1 added support for big-endian. – Donal Sep 25 '14 at 09:49
  • I hope you're not doing this to try to save space. – Rawling Sep 25 '14 at 09:53
  • If space is your concern - you could consider things like "varint"; this takes 1 byte for 0-127; 2 bytes for 128-16,383; 3 bytes for 16,384 to 2,097,151; 4 bytes for 2,097,152 to 268,435,455; and 5 bytes for 268,435,456 to 4,294,967,295 (assuming unsigned, etc) - and has defined endianness (/cc @Rawling) (actually, 5 bytes takes you all the way to 34,359,738,367 - if you really need it) – Marc Gravell Sep 25 '14 at 10:01
  • @MarcGravell - the only reference to I can find to varint is this [link]http://pastebin.com/Qk7LXESQ? – winnt93 Sep 25 '14 at 10:32
  • 1
    @winnt93 it is an encoding detail from "protocol buffers"; here: https://developers.google.com/protocol-buffers/docs/encoding#varints - basically, it uses 7 bits data, and the 8th bit as a continuation flag; [example reader](https://github.com/mgravell/protobuf-net/blob/master/protobuf-net/ProtoReader.cs#L1105-L1136); [example writer](https://github.com/mgravell/protobuf-net/blob/master/protobuf-net/ProtoWriter.cs#L539-L552) – Marc Gravell Sep 25 '14 at 11:27

2 Answers2

2

It entirely depends on what you are doing with the data. If you are going to be writing it to disk for portable persistence, then yes... I would probably care about endianness. If you are just going to use it to recreate an int later in the same process (or on the same machine), it probably doesn't matter as much.

However, when I do need to worry about endianness, I usually don't acheive that by BitConverter at all - personally, I'd be tempted to use byte masking and shifting; then you don't even need to know the endianness - it'll work the same on any system. It also avoids the annoyingly bad design decision of BitConverter returning a byte array rather than accepting an array and offset.

For example:

byte[] buffer = ...

// write little-endian
buffer[offset++] = (byte)(i & 0xFF);
buffer[offset++] = (byte)((i >> 8) & 0xFF);
buffer[offset++] = (byte)((i >> 16) & 0xFF);
buffer[offset++] = (byte)((i >> 24) & 0xFF);
Marc Gravell
  • 1,026,079
  • 266
  • 2,566
  • 2,900
  • Might be getting a little bit of off-topic here, but why are you saying it's a bad decision to create and return the array rather than accept it? – Theodoros Chatzigiannakis Sep 25 '14 at 09:52
  • 1
    @TheodorosChatzigiannakis unnecessary array allocations; in most scenarios where you want the bytes of a value, you are already working with local buffers - it would be nice to be able to use: `BitConverter.GetBytes(i, buffer, offset);` – Marc Gravell Sep 25 '14 at 09:53
  • 1
    `It entirely depends on what you are doing with the data` - well, he's trying to remove redundant `0` bytes, so I'm pretty sure he *should* be caring about endianness. – Rawling Sep 25 '14 at 09:55
2

Yes, according to the documentation, you need to be concerned. They have an example where they reverse the bytes if the architecture is not the desired endianess.

As far as where to get a BigEndian system, I think that the ARM based processors are big-endians, although I haven't tested this. So if you're running on Win RT device or a phone, for example, you might get different behavior.

FarmerBob
  • 1,314
  • 8
  • 11
  • Itaniums are big-endian too, although IIRC they can run individual processes litle-endian if needed, and again IIRC: when running .NET they always end up being little-endian – Marc Gravell Sep 25 '14 at 09:55
  • This basically answered my question - yes I need to be concerned and I'd missed that bit in the MSDN docs (so I now have a check and an array.reverse when required). However, @MarcGravell made reference to varint, which was also helpful. – winnt93 Sep 25 '14 at 13:12