1

I'm getting some sort of encoded data from server, which should be properly decoded and displayed on client as string(UTF-16). By the moment I'm doing something like this:

_decodeContent: function(encodedContent) {
  var binaryContent = atob(encodedContent);
  var result = pako.inflate(binaryContent, { to: 'string' });

  return result;
}

In most cases it works fine. But sometimes client have to handle massive amount data, so it freezes(Google Chrome 51.0.2704.84) or even crashes(Chromium 50.0.2661.102) browser tab. So I've find out, that problems is { to: 'string' } option in pako's inflate method, while trying to convert Uint8Array containing about 50kk elements to UTF-16 string.

I've tried to convert this manualy in a way like this:

_decodeContent: function(encodedContent) {
  var binaryContent = atob(encodedContent);
  var utf8 = pako.inflate(binaryContent);
  var result = String.fromCharCode.apply(null, utf8);

  return result;
}

But it produces: Uncaught RangeError: Maximum call stack size exceeded

So now I'm looking for solution to make things work faster, or at least prevent browsers from crashing. Any ideas?

UPD: Tried to do String.fromCharCode.apply with chunks to prevent the exception. So problem with the exception was solved, but it works about two times slower than very first solution.

UPD: So the problem is how to convert massive Uint8Array to UTF-16 string preventing browser from freezing.

qwr
  • 9,525
  • 5
  • 58
  • 102
Filipp Shestakov
  • 651
  • 7
  • 17
  • What exactly is `encodedContent`? Why not just send the correct encoding HTTP headers and let the browser do the necessary decoding for you? – Bergi Jul 01 '16 at 12:44
  • Originally it is GZIP Base64 string. So, I don't thinks its that simple. – Filipp Shestakov Jul 01 '16 at 12:59
  • Why base64? Just gzip it and send that with `Content-Encoding: gzip`. – Bergi Jul 01 '16 at 13:05
  • Unfortunately, affecting on this data encoding is out of my competence. I just have to find out possible way to handle such massive data on client side. – Filipp Shestakov Jul 01 '16 at 13:13
  • And by the way, server still uses UTF-8 version of this string. So that would not solve the problem. – Filipp Shestakov Jul 01 '16 at 13:21
  • You can prevent the exception by splitting up the array in reasonably-sized chunks and using `fromCharCode.apply` on them separately, but whether that would help your browser from being crushed by the amount of data I cannot say. – Bergi Jul 01 '16 at 13:23
  • That's wierd but `fromCharCode.apply` with chunks is about two times slower, than `pako.inflate(binaryContent, { to: 'string' })`. – Filipp Shestakov Jul 01 '16 at 13:58

1 Answers1

1

I avoided Firefox 86 browser crash on 100 MB string using TextDecoder(), no chunking required. YMMV on larger strings.

Update: the suggestion is implemented into the library (issue #228)

If you can avoid it, use browser's and server's builtin Content-Encoding: gzip, which will be faster and save you a library to import.

qwr
  • 9,525
  • 5
  • 58
  • 102