Variable-sized bitfields with aliasing

Question

I have some struct containig a bitfield, which may vary in size. Example:

struct BitfieldSmallBase {
    uint8_t a:2;
    uint8_t b:3;
    ....
}

struct BitfieldLargeBase {
    uint8_t a:4;
    uint8_t b:5;
    ....
}

and a union to access all bits at once:

template<typename T>
union Bitfield 
{
    T bits;
    uint8_t all;    // <-------------  Here is the problem

    bool operator & (Bitfield<T> x) const {
        return !!(all & x.all);
    }
    Bitfield<T> operator + (Bitfield<T> x) const {
        Bitfield<T> temp;
        temp.all = all + x.all;   //works, because I can assume no overflow will happen
        return temp;
    }
    ....
}

typedef Bitfield<BitfieldSmallBase> BitfieldSmall;
typedef Bitfield<BitfieldLargeBase> BitfieldLarge;

The problem is: For some bitfield base classes, an uint8_t is not sufficient. BitfieldSmall does fit into a uint8_t, but BitfieldLarge does not. The data needs to be packed as tightly as possible (it will be handled by SSE instructions later), so always using uint16_t is out of question. Is there a way to declare the "all" field with an integral type, whose size is the same as the bitfield? Or another way to access bits as a whole?

I can of course forego the use of the template and declare every kind of bitfield explicitly, but I would like to avoid code repetition (there is quite a list of operators und member functions).

score 5 · Accepted Answer · answered Oct 01 '09 at 12:04

5

You could make the integral type a template parameter as well.

template<typename T, typename U>
union Bitfield 
{
    T bits;
    U all;
}

typedef Bitfield<BitfieldSmallBase, uint8_t>  BitfieldSmall;
typedef Bitfield<BitfieldLargeBase, uint16_t> BitfieldLarge;

answered Oct 01 '09 at 12:04

John Kugelman

349,597
67
533
578

Sometimes the solution is so simple :-) – Gunther Piez Oct 01 '09 at 12:06

score 1 · Answer 2 · answered Oct 01 '09 at 12:08

1

I've learnt the hard way that whilst the bit width on vars that you're using is a convenient way of getting the compiler to do your masking and shifting for you, you cannot make assumptions about the order and padding of the members in the struct. Its compiler dependent and the compiler really does change the order and such dependent upon the other code in your project.

If you want to treat a byte as discrete fields, you really have to do it the hard way.

answered Oct 01 '09 at 12:08

Will

73,905
40
169
246

Portability is secondary, because I need to use SSE anyway, so this is limited to x86_64 (and possibly x86). For the compilers I am using (gcc, icc) I can make safe assumptions about the layout of the structures, it is defined in the ABI. – Gunther Piez Oct 01 '09 at 12:31
but it is not guaranteed to be the same in the two compilers. Or in the *next* version of these compilers. – jalf Oct 01 '09 at 12:33
Both icc and gcc comply to the x86_64 ABI. There have been ABI breakages from time to time, but they usually try very hard to minimize the impact. If you find a case, where the structure layout of icc differs from gcc, you may even file a bug, because icc trys hard to be as gcc-compatible as possible. – Gunther Piez Oct 01 '09 at 12:58
Well it was GCC and x86_64 where I got the problems myself, and I think I was using 4.4.something. Just my data point. – Will Oct 01 '09 at 14:48
Can you elaborate where the problem was? I know bitfields are shunned by even some of the gcc developers, but sometimes they are handy for low level stuff – Gunther Piez Oct 02 '09 at 09:50

score 1 · Answer 3 · answered Oct 01 '09 at 12:08

1

you can use template metaprogramming to define a template function that maps from BitfieldSmallBase, BitfieldLargeBase, etc into another type - uint8_t by default and to uint16_t for BitfieldLargeBase as a template specialization and then use that like this:

union Bitfield 
{
    T bits;
    typename F<T>::holder_type all;
};

answered Oct 01 '09 at 12:08

catwalk

6,340
25
16

Another nice solution. But I prefer John Kugelmans solution, its even simpler. – Gunther Piez Oct 01 '09 at 12:16
John has a better solution if there are few of BitfieldSmall* classes; mine is better if there are many of them but one or two stand out and require, say uint16_t vs default uint8_t. – catwalk Oct 01 '09 at 12:30
There are 3 bitfield classes, one of them two-byte sized (so far..). Because the declaration of F::holder_type and its specialization is more than 3 lines, Johns solution is better in this case, albeit by a very slight margin ;-) – Gunther Piez Oct 01 '09 at 12:36

score 1 · Answer 4 · answered Oct 01 '09 at 12:09

1

You might want to consider std::bitset or boost::dynamic_bitset rather than rolling your own. In any case, steer clear of std::vector<bool>!

answered Oct 01 '09 at 12:09

D.Shawley

58,213
10
98
113

Thanks for the warning ;-) The arrays I use in this part of the program are very fixed in size (contrary to the underlying data structures), so I avoid std::vector anyway. And boost would not simplify things, because I need to access the data via SSE (__m128i and the like) to do the actual work. – Gunther Piez Oct 01 '09 at 12:23

score 0 · Answer 5 · answered Oct 01 '09 at 21:31

Make the number of bytes you need part of the template parameters:

template <typename T, int S=1>
struct BitField 
{
   union
   {
      T bits;
      unsigned char bytes[S];
   };
};

typedef Bitfield<BitfieldSmallBase, 1>  BitfieldSmall;
typedef Bitfield<BitfieldLargeBase, 2> BitfieldLarge;

score 0 · Answer 6 · answered Oct 01 '09 at 21:36

0

How about this?

#include <limits.h>

template <class T>
union BitField
{
    T bits;
    unsigned all : sizeof(T) * CHAR_BIT;
};

answered Oct 01 '09 at 21:36

Pavel Minaev

99,783
25
219
289

Variable-sized bitfields with aliasing

6 Answers6

Linked