Typed Arrays in Gecko 2: Float32Array concatenation and expansion

Question

I'm a bit confused with Javascript Typed Arrays.

What I have are several Float32Array s, that have no concat method. I don't know how many are them in advance, btw. I'd like to concatenate them all inside another Float32Array, but:

as I said before, there is no concatenation method
if I try to write past the array length, the array is not expanded (aka this won't work - please note that event.frameBuffer and buffer are both Float32Array and that I don't know what the final length of my buffer will be):

var length_now = buffer.length;
for (var i = 0; i < event.frameBuffer.length; i += 1) {
      buffer [length_now + i] = event.frameBuffer[i];
}

The only solution I found is to copy the Float32Array in a regular array, that's definitely not what I want. How would you do, stackoverflowers?

score 27 · Accepted Answer · edited Jan 26 '21 at 14:41

27

Typed arrays are based on array buffers, which cannot be resized dynamically, so writing past the end of the array or using push() is not possible.

One way to achieve what you want would be to allocate a new Float32Array, large enough to contain both arrays, and perform an optimized copy:

function Float32Concat(first, second)
{
    var firstLength = first.length,
        result = new Float32Array(firstLength + second.length);

    result.set(first);
    result.set(second, firstLength);

    return result;
}

That would allow you to write:

buffer = Float32Concat(buffer, event.frameBuffer);

edited Jan 26 '21 at 14:41

Zsolt Meszaros

21,961
19
54
57

answered Dec 29 '10 at 13:18

Frédéric Hamidi

258,201
41
486
479

1

This is really great. Two questions: continuosly re-create a new typed array won't impact performances? and where did you find documentation about the .set function member? It's not in the page you linked. – janesconference Dec 29 '10 at 14:15
@janesconference, well, it won't necessarily impact performance since `set()` is probably implemented natively and, as such, blindingly fast with memory blits, but it will impact memory since you can't just extend an existing type array. Depending on array size, if memory becomes scarce, thrashing might occur and performance will degrade tremendously as a result. – Frédéric Hamidi Dec 29 '10 at 14:30
1

@FrédéricHamidi: There is another issue than "native" implementation: Assume that you have n arrays with m elements you want to concatenate. Your complexity is then O(m^2), since you will copy larger and larger data blocks. The optimal solution is amortized O(m). – user877329 Oct 06 '17 at 09:15
2

if anyone comes across this as I did wondering about the answer to @janesconference's 2nd question - where is there documentation for `.set()` - see https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Float32Array – keymap Aug 19 '21 at 20:59
buffer = Float32Array.from([...buffera, ...bufferb]). spread operator works just fine no need to write anything. – PeterT Mar 01 '23 at 20:50

amwinter · Answer 2 · 2013-11-21T04:00:20.897

Or if you're trying to join N arrays:

// one-liner to sum the values in an array
function sum(a){
  return a.reduce(function(a,b){return a+b;},0);
}

// call this with an array of Uint8Array objects
function bufjoin(bufs){
  var lens=bufs.map(function(a){return a.length;});
  var aout=new Uint8Array(sum(lens));
  for (var i=0;i<bufs.length;++i){
    var start=sum(lens.slice(0,i));
    aout.set(bufs[i],start); // copy bufs[i] to aout at start position
  }
  return aout;
}

Chad Cache · Answer 3 · 2016-07-14T02:23:39.833

I had the same issue, you can add the following to the prototype

Float32Array.prototype.concat = function() {
    var bytesPerIndex = 4,
        buffers = Array.prototype.slice.call(arguments);

    // add self
    buffers.unshift(this);

    buffers = buffers.map(function (item) {
        if (item instanceof Float32Array) {
            return item.buffer;
        } else if (item instanceof ArrayBuffer) {
            if (item.byteLength / bytesPerIndex % 1 !== 0) {
                throw new Error('One of the ArrayBuffers is not from a Float32Array');  
            }
            return item;
        } else {
            throw new Error('You can only concat Float32Array, or ArrayBuffers');
        }
    });

    var concatenatedByteLength = buffers
        .map(function (a) {return a.byteLength;})
        .reduce(function (a,b) {return a + b;}, 0);

    var concatenatedArray = new Float32Array(concatenatedByteLength / bytesPerIndex);

    var offset = 0;
    buffers.forEach(function (buffer, index) {
        concatenatedArray.set(new Float32Array(buffer), offset);
        offset += buffer.byteLength / bytesPerIndex;
    });

    return concatenatedArray;
};

now you can simply do

var array1 = new Float32Array(10000000),
    array2 = new Float32Array(10000000);

var array3 = array1.concat(array2);

Typed Arrays in Gecko 2: Float32Array concatenation and expansion

3 Answers3

Linked