Implement existing network interface defining bit-fields with respect to endianess in C++11

Question

I am current developing a C++11 library for an existing network remote control interface described in an Interface Control Document (ICD).

The interface is based on TCP/IPv4 and uses Network Byte Order (aka Big Endian).

Requirement: The library shall be developed cross-platform.

Note: I want to develop a solution without (ab)using the preprocessor.

After a short research on the WWW I discovered Boost.Endian which solves problems related to endianess of multi-byte data types. My approach is as follows:

1. Serialize the (multi-)byte data types to a stream via std::basic_ostream::write, more precisely via os.write(reinterpret_cast<char const *>(&kData), sizeof(kData)).
2. Convert the std::basic_ostream to a std::vector<std::uint8_t>.
3. Send the std::vector<std::uint8_t> via Boost.Asio over the network.

So far so good. Everything seems to work as intended and the solution should be platform independent.

Now comes the tricky part: The ICD describes messages consisting of multiple words and a word consists of 8 bits. A message can contain multiple fields and a field does not have to be byte-aligned, which means that one word can contain multiple fields.

Example: Consider the following message format (the message body starts at word 10):

Word | Bit(s) | Name
-----|--------|----------
 10  |  0-2   |  a
 10  |   3    |  b
 10  |   4    |  c
 10  |   5    |  d
 10  |   6    |  e
 10  |   7    |  RESERVED
 11  |   16   |  f

and so on...

So now I need a solution to be able to model and serialize a bit-based interface.

I have looked at the following approaches so far:

1. Bit field
2. std::bitset

3. boost::dynamic_bitset

   • 1 is not cross-platform (compiler dependent).
   • 2 and 3 do seem to work with Native Byte Order (i.e. Little Endian on my machine) only, so the following example using boost::dynamic_bitset does not work for my scenario:

Code:

// Using a arithmetic type from the `boost::endian` namespace does not work.
using Byte = std::uint8_t;
using BitSet = boost::dynamic_bitset<Byte>;
BitSet bitSetForA{3, 1};
BitSet bitSetForB{1};
// [...]
BitSet bitSetForF{16, 0x400}; // 1024

So, the 1024 in the example above is always serialized to 00 04 instead of 04 00 on my machine.

I really do not know what's the most pragmatic approach to solve my problem. Maybe you can guide me into the correct direction.

In conclusion I do need a recipe to implement an existing network interface defining bit fields in a platform-independent way with respect to the native byte order of the machine the library has been compiled on.

Answer 1

Someone recently kindly pointed me to a nice article about endianness from which I took inspiration.

std::bitset has a to_ulong method that can be used to return the integer representation of the bitfield (endian independent), and the following code will print your output in the correct order:

#include <iostream>
#include <iomanip>
#include <bitset>

int main()
{
  std::bitset<16> flags;
  flags[10] = true;
  unsigned long rawflags = flags.to_ulong();
  std::cout << std::setfill('0') << std::setw(2) << std::hex
            << (rawflags & 0x0FF) // little byte in the beginning                                                     
            << std::setw(2)
            << ((rawflags>>8) & 0x0FF) // big byte in the end                                                         
            << std::endl;
}

Note that no solution using bit fields will easily work in this case, since also the bits will be swapped on little endian machines!

e.g. in a structure like this:

struct bits {
  unsigned char first_bit:1;
  unsigned char rest:7;
};
union {
  bits b;
  unsigned char raw;
};

Setting b.fist_bit to 1 will result in a raw value of 1 or 128 depending on the endianness!

HTH