Java - optimize writing values as bits to bytebuffer

Question

I am currently working on some networking code (this is my first server) and had a quick question about optimizing a specific function which writes values as bits and then packs them into a byte. The reason for optimizing this function is because it is used thousands of times each server tick for packing data to be sent to several clients.

An example may serve better in order to explain what the function tries to accomplish: The value 3 can be represented by two bits. In binary, it would look like 00000011. The function would turn this binary value into 11000000. When the function is called again, it would know to start from the 3rd most significant bit (3rd from the right/decimal 32) and write at most up to 6 bits into the current byte. If then there were remaining bits to write, it would start on a new byte.

The purpose of this is to save space if you have multiple values that can be less than byte.

My current function looks like this:

 private ByteBuffer out = ByteBuffer.allocate(1024);
 private int bitIndex = 0;
 /*
  * Value: The value to write
  * Amount: The number of bits to represent the value in.
  */
     public OutputBuffer writeBits(long value, int amount) {
    if (bitIndex != 0) {
        int remainingBits = 8 - bitIndex;
        int bytePos = out.position() - 1;
        byte current = out.get(bytePos);
        int shiftAmount = amount - remainingBits;
        int bitsWritten = amount < remainingBits ? amount : remainingBits;
        int clearShiftAmount = 8 - bitsWritten + 56;

        byte b;
        if (shiftAmount < 0) {
            b = (byte) (current | (value << remainingBits - amount));
        } else {
            //deal with negative values
            long temp = (value >> shiftAmount);
            temp =  (temp << clearShiftAmount);
            temp = (byte) (temp >>> clearShiftAmount);
            b = (byte) (current | temp);
        }
        out.put(bytePos,b);
        bitIndex = (bitIndex + bitsWritten) % 8;
        amount -= bitsWritten;
    }
    if (amount <= 0) {
        return this;
    }
    bitIndex = amount & 7;
    int newAmount = amount - bitIndex;
    //newValue should not equal 2047
    for (int i = 0; i != newAmount; i += 8) {
        writeByte((byte) ((value >> i)), false);
    }
    if (bitIndex > 0)
        writeByte((byte) (value << (8 - bitIndex)), false);
    return this;
}

As I'm new to this I think there may be more efficient ways, maybe using bit-masking or a lookup table of some sort? Any ideas or steering in the right direction would be great. Cheers.

Answer 1

Ok, I tweaked your original algorithm to remove some of the redundant math, and I shaved about 10% off (went from 0.016 ms to about 0.014 ms on my machine). I also altered my test to run each algorithm 1000 times.

There also appears to be some savings to be had in that last for loop because the same bits are being shifted over and over. If you could somehow retain the results of the previous shift that might help. But that would alter the order of bytes going to out, so would require some more thought.

public void writeBits3(long value, int amount) {
    if (bitIndex != 0) {
        int remainingBits = 8 - bitIndex;
        int bytePos = out.position() - 1;
        byte current = out.get(bytePos);
        int shiftAmount = amount - remainingBits;

        int bitsWritten = 0;
        if (shiftAmount < 0) {
            bitsWritten = amount;
            out.put(bytePos, (byte) (current | (value << -shiftAmount)));
        } else {
            bitsWritten = remainingBits;
            out.put(bytePos, (byte) (current | (value >> shiftAmount)));
        }

        bitIndex += bitsWritten;
        amount -= bitsWritten;
        if (bitIndex >= 8) {
            bitIndex = 0;
        }
    }
    if (amount <= 0) {
        return;
    }
    bitIndex = amount & 7;
    int newAmount = amount - bitIndex;
    long newValue = (value >> bitIndex);
    for (; newAmount >= 8; newAmount -= 8) {
        out.put((byte) (newValue >> newAmount));
    }
    out.put((byte) (value << (8 - bitIndex)));
}

Answer 2

Here's a better solution (than my previous one) using recursion, which is perfect for this problem.

private static final long[] mask = { 0, 0x1, 0x3, 0x7, 0xf, 0x1f, 0x3f, 0x7f, 0xff };

private ByteBuffer out = ByteBuffer.allocate(1024);
private int position = 0;
private int dataBits = 0;
private byte remainder = 0;

/**
 * value: The value to write
 * amount: The number of bits to represent the value in.
 */
public void writeBits(long value, int amount) {
    if (amount <= Long.SIZE) {
        if (amount > 0) {
            // left align the bits in value
            writeBitsLeft(value << (Long.SIZE - amount), amount);
        } else {
            // flush what's left
            out.put(position++, remainder);
        }
    } else {
        // the data provided is invalid
        throw new IllegalArgumentException("the amount of bits to write is out of range");
    }
}

/**
 * write amount bits from the given value
 * 
 * @param value represents bits aligned to the left of a long
 * @param amount bits left to be written from value
 */
private void writeBitsLeft(long value, int amount) {
    if (amount > 0) {
        // how many bits are left to be filled in the current byte?
        int room = Byte.SIZE - dataBits;

        // how many bits are we going to add to the current byte?
        int taken = Math.min(amount, room);

        // rotate those number of bits into the rightmost position
        long temp = Long.rotateLeft(value, taken);

        // add count taken to the count of bits in the current byte
        dataBits += taken;

        // add in that number of data bits
        remainder &= temp & mask[taken];

        // have we filled the byte yet?
        if (Byte.SIZE == dataBits) {
            out.put(position++, remainder);

            // reset the current byte
            remainder = 0;
            dataBits = 0;

            // process any bits left over
            writeBitsLeft(temp, amount - taken);
        }
    } 
} // writeBitsLeft()

This solution has fewer math operations, fewer shift operations and fewer if statements and therefore should be more efficient than the original solution, not to mention it may be a little easier to understand.

Answer 3

What about something like this?

private ByteBuffer out = ByteBuffer.allocate(1024);
private int position = 0;
private int dataBits = 0;
private long data = 0;

/**
 * value: The value to write
 * amount: The number of bits to represent the value in.
 */
public void writeBits(long value, int amount) {
    if (amount <= 0) {
        // need to flush what's left
        if (dataBits > 0) {
            dataBits = Byte.SIZE;
        }
    } else {
        int totalBits = dataBits + amount;

        // need to handle overflow?
        if (totalBits > Long.SIZE) {
            // the new data is to big for the room that remains;  by how much?
            int excess = totalBits - Long.SIZE;

            // drop off the excess and write what's left
            writeBits(value >> excess, amount - excess);

            // now we can continue processing just the rightmost excess bits
            amount = excess;
        }

        // push the bits we're interested in all the way to the left of the long
        long temp = value << (Long.SIZE - amount);

        // make room for any existing (leftover) data bits, filling with zeros to the left (important)
        temp = temp >> dataBits;

        // append the new data to the existing
        data |= temp;

        // account for new bits of data
        dataBits += amount;
    }

    while (dataBits >= Byte.SIZE) {
        // shift one byte left, rotating the byte that falls off into the rightmost byte
        data = Long.rotateLeft(data, Byte.SIZE);

        // add the rightmost byte to the buffer
        out.put(position++, (byte)(data & 0xff));

        // drop off the rightmost byte
        data &= 0xffffffffffffff00L;

        // setup for next byte
        dataBits -= Byte.SIZE;
    }
}