1

I am using protobufjs to manage my data over an API. It works perfectly most of the time but in one instance of a message I get an 'illegal buffer' exception. This is thrown by some internal code in the library. I am pasting here the Chrome debugger visual while stopped at a breakpoint se to the throw statement. Chromer debugger output

As you can see, Chrome tells me that the buffer is indeed an Uint8Array (of 755 bytes). Why the if statement resolved into false and cause the throw statement to be executed? Both "buffer instanceof Uint8Array" and "Array.isArray(buffer)" are true.

UPDATE

I wrote some code (eseentailly copied it from protobufjs and simplified it):

function test() {
    var data = new Uint8Array([10, 9, 18, 7, 99, 111, 110, 110, 101, 99, 116, 16, 1]);
    testProtobuf(data);
}

function Reader(buffer) {
    this.buf = buffer;
    this.pos = 0;
    this.len = buffer.length;
}

var create_array = function (buffer) {
    console.log(buffer);
    if (buffer instanceof Uint8Array || Array.isArray(buffer))
        return new Reader(buffer);
    throw Error("illegal buffer");
}

function testProtobuf (data) {
    try {
        create_array(data);
    }
    catch (e) { console.log('Exception')};
}

When I call test(), which calls testProtobuf, which in turn calls create_array no exception is thrown. I also call testProtobuf from my onmessage method in the real code. In this case, an exception is still thrown. As you can see, I log the buffer on the console. Both logs are identical (I made sure the test data was the same).

Here is the Chrome console: Console output in Chrome

Claude
  • 51
  • 2
  • 7
  • Any chance it's a `Uint8Array` instance from another realm (frame etc)? – Bergi Aug 15 '20 at 16:20
  • "*Both "buffer instanceof Uint8Array" and "Array.isArray(buffer)" are true.*" - well, apparently they are not, why did you make this assumption? – Bergi Aug 15 '20 at 16:21
  • Another realm? What do yo mean? For the assumption: I am making it because the exception is thrown (the breakpoint at the throw staement is where Chronme stopped), whereas for other message it is not thrown. Clearly this is related to the buffer or its data but I can not figure it out. – Claude Aug 15 '20 at 17:11
  • Can you make a [mcve] that demonstrates the problem? – Bergi Aug 15 '20 at 17:12
  • Updated the main quiestion to add some code. The test code, hower, does not fail, albeit the data is identical to when the code fails. – Claude Aug 15 '20 at 22:17
  • Sorry, we can't help you if you can't show the failing code – Bergi Aug 15 '20 at 22:20
  • The difficulty is that the Rx processing involves decryption; I took the output of the decryptor for the messages that fail and pass it through the code I posted and that failed too. I also manually built a 13B buffer in the sample code identical to 1 of the fail message. That did not fail. The difference seems to be in the 2 Uint8Array. Beside printing the data on the console, which shows that the 2 buffer are identical, how can I compare the 2 buffers? Understanding the difference between the 2 buffers appears to me easier than understanding where the difference comes from. – Claude Aug 15 '20 at 23:07
  • Check whether `brokenBuffer.constructor.name == 'Uint8Array'` and `brokenBuffer.constructor === Uint8Array` – Bergi Aug 15 '20 at 23:18
  • The names are the same and match 'Uint8Array'. The 2nd fails with the broken buffer. But is obvisously Ok for the good buffer. I tried to find any differences, paying attention to the object hierachy but both look the same, I am sure there is something hidden somewhere. – Claude Aug 15 '20 at 23:39
  • That seems to support my initial hunch that the broken buffer comes from a different environment (with separate globals) such as an iframe. You should be able to convert it into a buffer from the current realm by using `Uint8Array.from(broken)` or maybe even `new Uint8Array(broken.buffer)`. – Bergi Aug 16 '20 at 00:00
  • Yeap, that works. The different environment is actually a different tab (window.open) where the master provides a reference to its session object (where encryption and decryption takes place and hence buffers are created). I guess I need to read more about 'realms'... since the 2 tabs share the same browser thread (a break point in 1 tab stops the execution in the other) I was under the impression that the 2 environments also shared the same memory space. – Claude Aug 16 '20 at 00:21
  • Same thread and same memory, yes, but there are two sets of builtins, making `instanceof` checks problematic. – Bergi Aug 16 '20 at 00:23

2 Answers2

0

I found a solution for this problem here:

Sample.deserializeBinary(Array.from(buffer));

And there is an example from my typescript code:

const arrayBuffer = await res.arrayBuffer();
const array = new Uint8Array(arrayBuffer);
const response: Uint8Array = (Array.from(array) as unknown) as Uint8Array;
rpcImplCallback(null, response);
Alex Po
  • 1,837
  • 1
  • 24
  • 28
0

You need to massage the response a little bit before it's ready to be tested/decoded ..

const lResponse = await res.arrayBuffer();
const lResponseData = ProtoModel.decode(new Uint8Array(lResponse));
Andre Thompson
  • 300
  • 2
  • 9