10

I need to store values in a range of about -10000 to 10000. At the range borders, the precision needed is not very high (maybe about 64, I will see how it fits), around zero the precision needs to be 1 or better.

Unfortunately space is very limited, not more than 12 bits, less would even be better. So strictly speaking, even half floats are out. Is there an open source library which handles some very short float formats with short mantissa and exponent length? Like 8 bit mantissa and 3 bit exponent.

Only conversion from/to bigger formats is required, no arithmetic is done.

Gunther Piez
  • 29,760
  • 6
  • 71
  • 103
  • With a 3 bits exponent you would have a 128 minimum precision. – xanatos Mar 15 '12 at 13:42
  • Yes, most likely I will end up cooking something by myself using bitfields. But I am interested in seeing the solutions others had found - if there are any. – Gunther Piez Mar 15 '12 at 13:43
  • @xanatos: only if you stick to a radix of 2. If you switch to a radix of 10, he only needs 9 bit mantissa, 2 bit radix. (Accuracy of 39 near 10000) – Mooing Duck Mar 15 '12 at 18:16
  • @drhirsch: just occurred to me, you can have a library that only uses 12 bits of precision, but you can't store them as is, as it will be padded (probably to 16). You'll have to bit-twiddle to store it in a 12 bit field. – Mooing Duck Mar 15 '12 at 18:21
  • @MooingDuck I am fully aware of that :-) I store several of these values in one bucket, which is aligned at a byte address. – Gunther Piez Mar 15 '12 at 22:45
  • This question would also be welcome on [scicomp](http://scicomp.stackexchange.com) – Aron Ahmadia Mar 17 '12 at 19:30

4 Answers4

4

If you're not doing any computation, then there's barely a need for a library. frexp, ldexp and some bit-twiddling should do the job.

Oliver Charlesworth
  • 267,707
  • 33
  • 569
  • 680
3

Perhaps μ-law or A-law could be useful for you.

Jens Björnhager
  • 5,632
  • 3
  • 27
  • 47
  • 1
    oh, interesting thought. You should expand on this answer. A lot. – Mooing Duck Mar 15 '12 at 17:46
  • If he is precise to nearest 1 from 0-12, precision 2 from 13-25, precision 3 from 26-38... thats precision 39 from 9671-10139, and it fits in 9 bits! (leaving 10th bit for sign) I have no idea how to do the math for this though. – Mooing Duck Mar 15 '12 at 18:24
  • Very interesting. I did implement something smiliar with only 4 ranges in my old code. Now I will try to find an efficient generalization or abstraction and see what ratio of ranges/values fits best. – Gunther Piez Mar 15 '12 at 19:01
  • There is a lot you can try with (natural) logs, but this may be a good place to start. – Jens Björnhager Mar 16 '12 at 13:22
2

We use libHalf which comes with openexr. I am not a big fan of it since the code quality isn't exactly stellar (though it also isn't seriously broken). Look for the directory named Half in the extracted sources -- it should be standalone.

Benjamin Bannier
  • 55,163
  • 11
  • 60
  • 80
1

Extrapolating from Jens Björnhager's answer, I got this:

double from_storage(unsigned short bits) { //only lowest 10 bits read
        double a = bits&0x1FF;
        a /= 5.11967985;
        a = a*a;
        return double(bits&0x200 ? -a : a);
}

unsigned short to_storage(double a) { //only lowest 10 bits set/relevant
        assert(a<=10000.0);
        assert(a>=-10000.0);
        if (a >= 0) {
                a = std::pow(a, .5); // inverse of number in from_storage
                a *= 5.11967985;
                unsigned short b = ((unsigned short)(a));
                assert((b&0x200)==0);
                return b;
        } else {
                a = std::pow(-a, .5);
                a *= 5.11967985;
                unsigned short b = ((unsigned short)(a));
                assert((b&0x200)==0);
                return b | 0x200;
        }
}

as demonstrated at http://ideone.com/DLTUn. This can hold each of the values between -10 and 10 uniquely, and the top values are only 39 apart. (There's also 3 values between 0 and 1). Someone more mathy could probably get the positive and negative to more of a "two's compliment" format, which would cut the to_storage code in half.

Mooing Duck
  • 64,318
  • 19
  • 100
  • 158
  • 1
    Thank you for your input. I think I will use some variant of µ-Law (which is very similar to a quarter float) extended to 11 or 12 bits. I need to store several 10^9 of these values (the more, the better), thats the main reason I want to save every bit. Speed of conversion is another concern - pow is most likely to slow, but some table driven approach will do. – Gunther Piez Mar 15 '12 at 22:42
  • @drhirsch: this works along the same lines as µ-Law, except with a smooth curve. A table based approach might be faster, but would be more complicated. I think the table is the better answer in all. – Mooing Duck Mar 15 '12 at 23:16