Converting floats to strings in javascript

Question

Ok i want to save an array of THREE.Vector3 efficiently to local storage. Since javascript works using Strings, i want to convert a 32bit float to a string using the most efficient bit ratio. i.e ideally 32 bit float = 4 * 8 Bit which is easy to do in something like C++

the problem seems to be 1st Javascript strings are UTF which includes some padding

http://en.wikipedia.org/wiki/UTF-8

and secondly the code i am currently using 0 get converted '' and then omitted, making the converted byte length un-reliable.

String.fromCharCode(0) == ''

var float2str = function(num)
{
    var bytestream = new Array();
    var view = new DataView(new ArrayBuffer(4));
    view.setFloat32(0,num);

    bytestream.push(view.getUint8(0));
    bytestream.push(view.getUint8(1));
    bytestream.push(view.getUint8(2));
    bytestream.push(view.getUint8(3));
    return String.fromCharCode(view.getUint8(0),view.getUint8(1),view.getUint8(2),view.getUint8(3))

}

var str2float = function(str)
{

    var bytestream = unpack(str)
    var view = new DataView(new ArrayBuffer(4));
    view.setUint8(0,bytestream[0]);
    view.setUint8(1,bytestream[1]);
    view.setUint8(2,bytestream[2]);
    view.setUint8(3,bytestream[3]);
    return view.getFloat32(0);
}

thanx!

Why can't you store the floats simply as floats? I mean as far as I know localStorage should be able to store all JavaScript objects efficiently, regardless of their type. — jsalonen, Oct 16 '12 at 15:37
where on earth did you ear that javascript works only with strings? — , Oct 16 '12 at 15:43
yes i belive you have to store a string wiht local storage. if you store a number like Pi - 3.142.... each numeral is stored as 1 character which is massively inneficient — RenegadeMaster88, Oct 17 '12 at 14:11

score 1 · Answer 1 · answered Oct 16 '12 at 16:24

The trick is in getting the string value of the unsigned 8-bit integers to be printable. I found that if your 8-bit numbers are too small or too large (outside of the "bread and butter" range of ASCII values) you will end up with something that is unprintable. So instead of creating a string of length four using the four ASCII values of the bytes from the float, we can use a string of length 8 using 4-bit values from the float and offsetting those values into the printable range (+32). Here's my solution:

var offset = 33;

var float2str = function(num){
    var view = new DataView(new ArrayBuffer(4));
    view.setFloat32(0,num);

    var fourbits = [];
    for ( var i = 0; i < 4; i++ ){
        // split each byte into two 4-bit values
        fourbits.push(view.getUint8(0) >> 4);
        fourbits.push(view.getUint8(0) & 15);
    }

    for ( var i = 0; i < 8; i++ ){
        // build a string with offset four-bit values
        fourbits[i] = String.fromCharCode(offset+fourbits[i]);
    }
    return fourbits.join('');

};

var str2float = function(str){
    var bytestream = str;
    var view = new DataView(new ArrayBuffer(4));

    for(var i = 0; i < 4; i++){
        // re-convert the characters into bytes.
        view.setUint8(i, ((bytestream[i*2].charCodeAt() - offset) << 4) + bytestream[i*2+1].charCodeAt() - offset);
    }

    return view.getFloat32(0);
};

console.log(float2str('2.251')); // "%!\"!\"!'#"
console.log(str2float(float2str('2.251'))); // 2.250999927520752

I hope this helps.

I used `33` as the offset here, so that we didn't get empty spaces in our output string, but `32` would be fine as well. — JasonWyatt, Oct 16 '12 at 16:26
As the original goal was to pack the floats efficiently, using 8 bytes per float kind of defeats that purpose as the regular string representation would be shorter in many practical cases. (According to Wikipedia 32 bit float has precision for 6-9 significant digits, which probably translates to worst-case scenario of 11 bytes for a string (decimal point and minus added), but I'd guess typical numbers/enough precision could most likely be achieved with less (e.g. 2595e-8 = 7)). — Tapio, Oct 17 '12 at 11:46
Maybe the best bet for storing large amounts of numbers in local storage is to simply create the JSON string containing the numbers - and run BZip on it before saving it to local storage. — JasonWyatt, Oct 17 '12 at 13:46
Hi Jason, thank you for your help, whilst your code does work it packs a 32 bit to 8 16 bit chars, which ends up as 128bits / float. I would like to get that down to 64bits / float if possible. I have tried using LZ78 based Zip on JSON strings, but they still end up being way larger than necessary even with a 10:1 compression ratio. — RenegadeMaster88, Oct 17 '12 at 14:26

score 0 · Answer 2 · answered Oct 17 '12 at 12:09

Use base64 encoding. It will convert 6 bits at a time to a printable ASCII character (which will be the same in UTF-8). As such, you will actually need 5.33 bytes for one float (the fraction is usable, i.e. it's not 6 bytes as long as you encode the array as a whole and not as individual floats), but that should still give some space savings.

Here's a function (not mine) to convert directly from ArrayBuffer to base64 string: https://gist.github.com/958841

Converting floats to strings in javascript

2 Answers2