C++ Compressing size of integer down to 2 bits?

Question

I am doing a little game physics networking project right now, and I am trying to optimize the packets I am sending using this guide:

https://gafferongames.com/post/snapshot_compression/

In the "Optimize Quaternions" section it says:

Don’t always drop the same component due to numerical precision issues. Instead, find the component with the largest absolute value and ENCODE its index using two bits [0,3] (0=x, 1=y, 2=z, 3=w), then send the index of the largest component and the smallest three components over the network

Now my question is, how do I encode an integer down to 2 bits... or have I misunderstood the task?

I know very little about compressing data, but reducing a 4 byte integer (32 bits) down to ONLY 2 bits seems a bit insane to me. Is that even possible, or have I completely misunderstood everything?

EDIT: Here is some code of what I have so far:

void HavNetConnection::sendBodyPacket(HavNetBodyPacket bp)
{
  RakNet::BitStream bsOut;
  bsOut.Write((RakNet::MessageID)ID_BODY_PACKET);

  float maxAbs = std::abs(bp.rotation(0));
  int maxIndex = 0;
  for (int i = 1; i < 4; i++)
  {
    float rotAbs = std::abs(bp.rotation(i));
    if (rotAbs > maxAbs) {
      maxAbs = rotAbs;
      maxIndex = i;
    }
  }

  bsOut.Write(bp.position(0));
  bsOut.Write(bp.position(1));
  bsOut.Write(bp.position(2));
  bsOut.Write(bp.linearVelocity(0));
  bsOut.Write(bp.linearVelocity(1));
  bsOut.Write(bp.linearVelocity(2));
  bsOut.Write(bp.rotation(0));
  bsOut.Write(bp.rotation(1));
  bsOut.Write(bp.rotation(2));
  bsOut.Write(bp.rotation(3));
  bsOut.Write(bp.bodyId.toRawInt(bp.bodyId));
  bsOut.Write(bp.stepCount);

  // Send body packets over UDP (UNRELIABLE), priority could be low.
  m_peer->Send(&bsOut, MEDIUM_PRIORITY, UNRELIABLE, 
      0, RakNet::UNASSIGNED_SYSTEM_ADDRESS, true);
}

Answer 1

I'm guessing they want you to fit the 2 bits into some value you are already sending that doesn't need all of the available bits, or to pack several small bit fields into a single int for transmission.

You can do things like this:

// these are going to be used as 2 bit fields,
// so we can only go to 3.
enum addresses
{
    x = 0,    // 00
    y = 1,    // 01
    z = 2,    // 10
    w = 3     // 11
};

int val_to_send;

// set the value to send, and shift it 2 bits left.
val_to_send = 1234;
// bit pattern:  0000 0100 1101 0010

// bit shift left by 2 bits
val_to_send = val_to_send << 2;
// bit pattern:  0001 0011 0100 1000

// set the address to the last 2 bits.
// this value is address w (bit pattern 11) for example...
val_to_send |= w;
// bit pattern: 0001 0011 0100 1011

send_value(val_to_send);

On the receive end:

receive_value(&rx_value);

// pick off the address by masking with the low 2 bits
address = rx_value & 0x3;
// address now = 3 (w)

// bit shift right to restore the value
rx_value = rx_value >> 2;
// rx_value = 1234 again.

You can 'pack' bits this way, any number of bits at a time.

int address_list;
// set address to w (11)
address_list = w;
// 0000 0011

// bit shift left by 2 bits
address_list = address_list << 2;
// 0000 1100

// now add address x (00)
address_list |= x;
// 0000 1100

// bit shift left 2 more bits
address_list = address_list << 2;
// 0011 0000

// add the address y (01)
address_list |= y;
// 0011 0001

// bit shift left 2 more bits
address_list = address_list << 2;
// 1100 0100

// add the address z. (10)
address_list |= z;
// 1100 0110
// w x  y z are now in the lower byte of 'address_list'

This packs 4 addresses into the lower byte of 'address_list';

You just have to do the unpacking on the other end.

This has some implementation details to work out. You only have 30 bits now for the value, not 32. If the data is a signed int, you have more work to do to avoid shifting the sign bit out to the left, etc.

But, fundamentally, this is how you can stuff bit patterns into data that you are sending.

Obviously this assumes that sending is more expensive than the work of packing bits into bytes and ints, etc. This is often the case, especially where low baud rates are involved, as in serial ports.

Answer 2

The simplest solution to your problem is to use bitfields:

// working type (use your existing Quaternion implementation instead)
struct Quaternion{
    float w,x,y,z;
    Quaternion(float w_=1.0f, float x_=0.0f, float y_=0.0f, float z_=0.0f) : w(w_), x(x_), y(y_), z(z_) {}
};

struct PacketQuaternion
{
    enum LargestElement{
        W=0, X=1, Y=2, Z=3,
    };
    LargestElement le : 2; // 2 bits;
    signed int i1 : 9, i2 : 9, i3 : 9;  // 9 bits each

    PacketQuaternion() : le(W), i1(0), i2(0), i3(0) {}

    operator Quaternion() const { // convert packet quaternion to  regular quaternion
        const float s = 1.0f/float(1<<8); // scale int to [-1, 1]; you could also scale to [-sqrt(.5), sqrt(.5)]
        const float f1=s*i1, f2 = s*i2, f3 = s*i3;
        const float f0 = std::sqrt(1.0f - f1*f1-f2*f2-f3*f3);
        switch(le){
        case W: return Quaternion(f0, f1, f2, f3);
        case X: return Quaternion(f1, f0, f2, f3);
        case Y: return Quaternion(f1, f2, f0, f3);
        case Z: return Quaternion(f1, f2, f3, f0);
        }
        return Quaternion(); // default, can't happen
    }
};

If you have a look at the assembler code this generates, you will see a bit of shifting to extract le and i1 to i3 -- essentially the same code you could write manually as well.

Your PacketQuaternion structure will always occupy a whole number of bytes, so (on any non-exotic platform) you will still waste 3 bits (you could just use 10 bits per integer field here, unless you have other use for those bits).

I left out the code to convert from regular quaternion to PacketQuaternion, but that should be relatively simple as well.

Generally (as always when networking is involved), be extra careful that data is converted correctly in all directions, especially, if different architectures or different compilers are involved!

Also, as others have noted, make sure that network bandwidth indeed is a bottle neck before doing aggressive optimization here.

Answer 3

There are a lot of possible understandings and misunderstandings in play here. ttemple addressed your technical problem of sending less than a byte. I want to reiterate the more theoretical points.

This is not done

You originally misunderstood the quoted passage. We do not use two bits to say “not sending 2121387”, but to say “not sending z-component”. That these match exactly, should be easy to see.

This is impossible

If you want to send a 32 bit integer which might take any of the 2^32 possible values, you need at least 32 bits. As n bits can represent at most exactly 2^n states, any smaller amount of bits just will not suffice.

This is kinda possible

Beyond your actual question: When we relax the requirement that we will always use 2 bits and have sufficiently strong assumptions on the probability distribution of the values, we can get the expected value of the number of bits down. Ideas like this are used all over the place in the linked article.

Example

Let c be some integer that is 0 almost all the time (97%, say) and can take any value the rest of the time (3%). Then we can take one bit to say whether “c is zero” and need no further bits most of the time. In the cases where c is not zero, we spend another 32 bits to encode it regularly. In total we need 0.97*1+0.03*(1+32) = 1.96 bits on average. But we need 33 bits sometimes, which makes this compatible with my earlier assertion of impossibility.

This is complicated

Depending on your background (in math, bit-fiddling etc.) it might just seem like an enormous, unknowable piece of black magic. (It isn't. You can learn this stuff.) You do not seem completely lost and a quick learner but I agree with Remy Lebeau that you seem to be out of your depth.

Do you really need to do this? Or are you optimizing prematurely? If it runs well enough, let it run. Concentrate on the important stuff.