Different ArrayBuffer for the same image (Creating ImageData in Worker problem)

Question

I'm trying to create ImageData inside the Worker by only posting the message to it with the image URL, width, and height. To make it work I need to fetch the image and get ArrayBuffer. With that I should be able to create new ImageData:

//worker.ts
fetch(e.data.url)
  .then( r => r.arrayBuffer() )
  .then( buffer => {
      console.log(buffer.byteLength); // 4022
      new ImageData((new Uint8ClampedArray(buffer)), e.data.width, e.data.height);
  });

I'm getting the following error: The input data byte length is not a multiple of (4 * width).

For Chrome I'm able to post to the Worker not just URL, but the whole ArrayBuffer created before in the main thread with canvas and that works:

//main thread
const canvas = typeof OffscreenCanvas !== 'undefined' ?
   new OffscreenCanvas(img.width, img.height) :
   document.createElement('canvas'),
   ctx = canvas.getContext('2d');

ctx.drawImage(img, 0, 0, img.width, img.height);
const imageData: ImageData = ctx.getImageData(0, 0, img.width, img.height);
console.log(imageData.data.buffer.byteLength); // 106288

The problem is that with Firefox worker.postMessage() can only handle smaller messages, like URL, not whole ArrayBuffer (takes a very long time to serialize and deserialize large messages). That's why I'm trying the approach with sending only URL, height, width, and getting the image in Worker itself. The problem is with ArrayBuffer fetched in the Worker - it's much smaller than the one created using canvas in the main thread (the same URL, different methods, different ArrayBuffer). If I would have the same ArrayBuffer in the Worker, ImageData would be created (not throwing error).

My question is what I'm getting wrong, how can I create ImageData after fetching image using fetch/axios, etc.

score 0 · Answer 1 · edited Jun 20 '20 at 09:12

An ImageData represents the Red Green Blue and Alpha values of what would become pixels when painted.

An image file on the other hand must have metadata. Even if the image is the most basic bitmap format you could invent, it would still need at least a width or height information.

So the file you are fetching is not just the pixel's RGBA channel's values, its a binary data, which need to be parsed, probably uncompressed, decoded, maybe even have some color-space processing before you can access these RGBA channels.

However, your premise that Firefox can't do this is flawed.

Don't clone that ArrayBuffer, transfer it, it takes no time.

const worker_script = document.querySelector( '[type="worker-script"]' );
const worker_content = worker_script.textContent;
const worker_blob = new Blob( [ worker_content ], { type: 'text/javascript' } );
const worker_url = URL.createObjectURL( worker_blob );
const worker = new Worker( worker_url );

let start;
worker.onmessage = e => {
  console.log( e.data );
  console.log( 'it took ' + (performance.now() - start) + 'ms to go from main thread to worker and back to main' );
};

const img = new Image();
img.onload = e => {
  const canvas = document.createElement( 'canvas' );
  canvas.width = img.width;
  canvas.height = img.height;
  const ctx = canvas.getContext( '2d' );
  ctx.drawImage( img, 0, 0, img.width, img.height );
  const imageData= ctx.getImageData( 0, 0, img.width, img.height );
  start = performance.now();
  worker.postMessage(
    imageData,
    [ imageData.data.buffer ] // transfer the ArrayBuffer
  );
};
img.crossOrigin = "anonymous";
img.src = "https://upload.wikimedia.org/wikipedia/commons/4/47/PNG_transparency_demonstration_1.png";

<script type="worker-script">
  onmessage = e => {
    const imageData = e.data;
    postMessage(
`worker received a ${ imageData.width } x ${ imageData.height } ImageData,
whose buffer's size is ${ imageData.data.buffer.byteLength }bytes`
    );
  };
</script>

Different ArrayBuffer for the same image (Creating ImageData in Worker problem)

1 Answers1

However, your premise that Firefox can't do this is flawed.