0

I have a formula in a specification for a binary file. The spec gives details of the meaning of the various bytes in the heading.
In particular, one formula states this about 2 of the bytes:

Byte 1  -->  7  6  5  4  3  2  1  0  
Byte 2  -->  7  6  5  4  3  2  1  0 

R  Roll 
 If 'R' = 0, Roll Angle not available 
 If 'R' = 1, Roll Angle = [((Byte 84 & 0x7F)<<8) | (Byte 85) – 900] / 10

I need to take a value such as 4.3 and convert it to two bytes such that it will be able to be converted back to 4.3 using the above formula. The part that puzzles me the most is subtracting the 900.

This is what I have so far:

private byte[] getRollBytes(BigDecimal[] positionData) {

    BigDecimal roll = positionData[4];
    roll = roll.multiply(BigDecimal.TEN);
    roll = roll.add(new BigDecimal(900));
    short shortval = roll.shortValue();

    byte[] rollBytes = new byte[2];

    ByteBuffer headingbuf = ByteBuffer.wrap(rollBytes);
    headingbuf.order(ByteOrder.BIG_ENDIAN);
    headingbuf.putShort(shortval);

    //set the leftmost bit of the two bytes to 'on', meaning data is available
    rollBytes[0] = (byte) (rollBytes[0] | (0x80));

    //testing the result with my formula doesn't give me the right answer:
    float testFloat = (float) (((((rollBytes[0] & 0x7F) <<8) | rollBytes[1]) - 900) /10);           

    return rollBytes;
}

I think something is getting lost in the conversion between short and byte...

GLaDOS
  • 683
  • 1
  • 14
  • 29

1 Answers1

0

One problem is that you are casting to a short. This can't be correct, because a value like 4.3 is not an integer. You probably want to convert back to a BigDecimal:

BigDecimal testVal = ((rollBytes[0] & 0x7F) <<8) | rollBytes[1]) - 900;
testVal = val.divide(BigDecimal.valueOf(10));

Another problem is that you seem to be adding 900 twice—once when computing shortVal and once again with rollBytes[1] = (byte) (rollBytes[1] + 900);

Ted Hopp
  • 232,168
  • 48
  • 399
  • 521
  • :$ Right, I forgot to take that out of my description... earlier I was trying to add 900 to a byte which didn't make sense... A friend pointed out the order of precedence of bitwise | and addition/subtraction means the bytes are concatenated into a 15-bit value before the subtraction/addition occurs, so working backwards I have to add that 900 before converting to a byte array. Ditto with the short. i'm using a float now. – GLaDOS Oct 04 '12 at 16:30
  • I had some goofiness going on with the way I was converting the bytes back to a decimal in order to test... and I found out I needed to round up the BigDecimal before converting to a short. This is what I get for trying to code when I have a cold. :-\ – GLaDOS Oct 04 '12 at 20:07