0

I need to extract some bit ranges from a 16-byte value, e.g.:

bit 0 = first thing
next 54 bits = second thing
next 52 bits = third thing
last 21 bits = fourth thing

.net doesn't have a UInt128 structure, well it has the BigInteger class, but I'm not sure that's right for the job, maybe it is?

I have found a third party library that can read bits from a stream, but when trying to convert them back to UInt64's using the BitConverter, it will fail, as 54 bits isn't long enough for a UInt64, but it's too long for a UInt32

My immediate thought was the bit shifting was the way to do this, but now I'm not so sure how to proceed, seeing as I can't think of a good way of handling the original 16 bytes.

Any suggestions or comments would be appreciated.

Tony
  • 3,587
  • 8
  • 44
  • 77
  • The brute force way to do this is to create a struct of the right size and then use masks (with the `&` operator) and shifts (with the `>>` and `<<` operators) to access properties. Where do you get this 16-bit value? – Flydog57 Apr 03 '19 at 15:46
  • The 16-bytes comes from a file. Yeah shifting it is fine, it's just getting the data into a shiftable state to begin with. – Tony Apr 03 '19 at 15:47
  • Use `[StructLayout(LayoutKind.Sequential)]` on a struct with two `UInt64`s. Then create properties for things 1..4 (and write code to do masking/shifting in the property getters/setters). The first, second and fourth are easy. For the third one, you'll need to dig the value out of the two ulongs and OR them together appropriate (the setters would reverse that process). It will be tedious to write, but pretty efficient once you get it all together. – Flydog57 Apr 03 '19 at 15:51
  • Thanks for the information. I've never done anything like you've suggested, I've only been using C# for six months or so. If you have time, would you mind producing a small amount of code, just to illustrate your point? Only if you have time of course, thanks. – Tony Apr 03 '19 at 15:54
  • By the way, doing this with an array of bytes (rather than a struct with either bytes or two ulongs) will cause you to tear your hair out. Structs are "blittable", arrays, not so much – Flydog57 Apr 03 '19 at 15:55
  • Fair point. I've not touched structs since my C days, and it's been a long time since then :) Thank you though. – Tony Apr 03 '19 at 15:57

1 Answers1

1

Here's some untested code. I'm sure that there are bugs in it (whenever I write code like this, I get shifts, masks, etc. wrong). However, it should be enough to get you started. If you get this working and there are only a few problems, let me know in the comments and I'll fix things. If you can't get it to work, let me know as well, and I'll delete the answer. If it requires a major rewrite, post your working code as an answer and let me know.

The other thing to worry about with this (since you mentioned that this comes from a file) is endian-ness. Not all computer architectures represent values in the same way. I'll leave any byte swizzling (if needed) to you.

First, structs in C++ are basically the same as classes (though people think they are different). In C#, they are very different. A struct in C# is a Value Type. When you do value type assignment, the compiler makes a copy of the value of the struct, rather than just making a copy to a reference to the object (like it does with classes). Value types have an implicit default constructor that initializes all members to their default (zero or null) values.

Marking the struct with [StructLayout(LayoutKind.Sequential)] tells the compiler to layout the members in the specified order (they compiler doesn't have to normally). This allows you to pass a reference to one of these (via P/Invoke) to a C program if you want to.

So, my struct starts off this way:

[StructLayout(LayoutKind.Sequential)]
public struct Struct128
{
    //not using auto-properties with private setters on purpose.
    //This should look like a single 128-bit value (in part, because of LayoutKind.Sequential)
    private ulong _bottom64bits;
    private ulong _top64bits;
}

Now I'm going to add members to that struct. Since you are getting the 128 bits from a file, don't try to read the data into a single 128-bit structure (if you can figure out how (look up serialization), you can, but...). Instead, read 64 bits at a time and use a constructor like this one:

 public Struct128(ulong bottom64, ulong top64)
 {
     _top64bits = top64;
     _bottom64bits = bottom64;
 }

If you need to write the data in one of these back into the file, go get it 64-bits at a time using read-only properties like this:

//read access to the raw storage
public ulong Top64 => _top64bits;
public ulong Bottom64 => _bottom64bits;

Now we need to get and set the various bit-ish values out of our structure. Getting (and setting) the first thing is easy:

public bool FirstThing
{
    get => (_bottom64bits & 0x01) == 1;
    set
    {
        //set or clear the 0 bit
        if (value)
        {
            _bottom64bits |= 1ul;
        }
        else
        {
            _bottom64bits &= (~1ul);
        }
    }
}

Getting/setting the second and fourth things are very similar. In both cases, to get the value, you mask away all but the important bits and then shift the result. To set the value, you take the property value, shift it to the right place, zero out the bits in the appropriate (top or bottom) value stored in the structure and OR in the new bits (that you set up by shifting)

//bits 1 through 55
private const ulong SecondThingMask = 0b111_1111_1111_1111_1111_1111_1111_1111_1111_1111_1111_1111_1111_1110;

public ulong SecondThing
{
    get => (_bottom64bits & SecondThingMask) >> 1;
    set
    {
        var shifted = (value << 1) & SecondThingMask;
        _bottom64bits = (_bottom64bits & (~SecondThingMask)) | shifted;
    }
}

and

 //top 21 bits
 private const ulong FourthThingMask = 0b1111_1111_1111_1111_1111_1000_0000_0000_0000_0000_0000_0000_0000_0000_0000_0000;
 //to shift the top 21 bits down to the bottom 21 bits, need to shift 64-21
 private const int FourthThingShift = 64 - 21;

 public uint FourthThing
 {
     get => (uint)((_top64bits & FourthThingMask) >> FourthThingShift);
     set
     {
         var shifted = ((ulong)value << FourthThingShift) & FourthThingMask;
         _top64bits = (_top64bits & (~FourthThingMask)) | shifted;
     }
 }

It's the third thing that is tricky. To get the value, you need to mask the correct bits out of both the top and bottom values, shift them to the right positions and return the ORed result.

To set the value, you need to take the property value, split it into upper and lower portions and then do the same kind of magic ORing that was done for the second and fourth things:

 //the third thing is the hard part.  
 //The bottom 55 bits of the _bottom64bits are dedicate to the 1st and 2nd things, so the next 9 are the bottom 9 of the 3rd thing
 //The other 52-9 (=43) bits come-from/go-to the _top64bits

 //top 9 bits
 private const ulong ThirdThingBottomMask = 0b1111_1111_1000_0000_0000_0000_0000_0000_0000_0000_0000_0000_0000_0000_0000_0000;
 //bottom 43 bits
 private const ulong ThirdThingTopMask = 0b111_1111_1111_1111_1111_1111_1111_1111_1111_1111_1111;
 private const int ThirdThingBottomShift = 64 - 9;

 //bottom 9 bits
 private const ulong ThirdThingBottomSetMask = 0b1_1111_1111;
 //all but the bottom 9 bits
 private const ulong ThirdThingTopSetMask = 0b1111_1111_1111_1111_1111_1111_1111_1111_1111_1111_1111_1111_1111_1110_0000_0000;
 //52 bits total
 private const ulong ThirdThingOverallMask = 0b1111_1111_1111_1111_1111_1111_1111_1111_1111_1111_1111_1111_1111;

 public ulong ThirdThing
 {
     get
     {
         var bottom = (_bottom64bits & ThirdThingBottomMask) >> ThirdThingBottomShift;
         var top = (_top64bits & ThirdThingTopMask) << 9;
         return top | bottom;
     }
     set
     {
         var masked = value & ThirdThingOverallMask;
         var bottom = (masked & ThirdThingBottomSetMask) << ThirdThingBottomShift;
         _bottom64bits = (_bottom64bits & (~ThirdThingBottomSetMask)) | bottom;
         var top = (masked & ThirdThingTopSetMask) >> 9;
         _top64bits = (_top64bits & (~ThirdThingTopSetMask)) | top;
     }
 }

I hope this is useful. Let me know.

Flydog57
  • 6,851
  • 2
  • 17
  • 18
  • Thanks very much for the code, I will take a look as soon as I can and report back. Appreciate the effort! – Tony Apr 04 '19 at 11:27
  • @Tony: You said "Thanks" two and a half years ago. Was this useful? If it was, why no up-vote or Accept for it? Someone finally up-voted it this week. Still waiting for feedback from the OP. As I mentioned, if you find any issues, let me know and I'll fix up the answer – Flydog57 Oct 25 '21 at 20:23
  • Sorry for the delay, time sure does fly! Thanks again – Tony Oct 26 '21 at 07:38