3

Im working on a image comparison web application with puppeteer and here's what im trying to do:

  1. open a puppeteer page, go to a rendering page with a model-viewer(a google's web graphic API) web component.
  2. render something with model-viewer
  3. take a rgba screenshot after the rendering is done (use page.screenshot())
  4. close the page.
  5. take the screenshot as an argument of the image comparison api (the input should be either nodejs Buffer type or a png file)

Everything works well except for the 3rd step, because by using page.screenshot() method, the output png file is in RGB format, not RGBA. I didn't find a way to do that and here're two solutions i tried:

Solution 1: model-viewer api has a build in function called toBlob(), which returns a Blob object of all the RGBA data of the model-viewer component i need. I know how to convert a Blob object to a Buffer so this should work if i get the Blob object. Here's how i try to get the blob object.

  const Blob = await page.evaluate(async () => {
    const modelViewer = document.querySelector('#modelViewer');
    const Blob = await (modelViewer as any).toBlob();
    //console.log(Blob) this shows a correct Blob object under puppteer browser's devtool

    // in order to return something, i have convert Blob object all the way to a unit8Array
    //const arrayBuffer = await new Response(Blob).arrayBuffer();
    //const unit8Array = new Uint8Array(arrayBuffer);

    return Blob;
  });

The problem is that the returned Blob object from page.evaluate() is empty. I doubled checked Blob object inside evaluate() function and it's logged correctly on puppteer browser's devtool. I think the reason is evaluate() requires the return value to be serializable while Blob object maybe not. In order to return something that's not empty to the application's context, i have to convert it all the way to a unit8Array. But i haven't found a way to convert unit8Array to a Buffer object and even if there is one, i think this can be very inefficient because the Blob object containing all pixels information can be really big and there probably will be a lot of computation in the conversion process.

Solution 2: Im thinking of using page.evaluateHandle() to get the jsHandle of the model-viewer component and run goBlob in my application's context, correct me if im wrong, but i think jsHandle are only supposed to use in puppteer's context, right? So this solution should be wrong

Therefore, is any of my solution can be improved? Or is there better solution? Thank you !

Jiaxin Sun
  • 71
  • 4

1 Answers1

1

If you want to stick with Puppeteer's screenshot tools, have you considered changing the page background and finding the alpha value algebraically? Placing the colour (r, g, b, a) on white gives an RGB value of (RW, GW, BW) = (ar + (1-a)255, ag + (1-a)255, ab + (1-a)255)), while placing it on black gives (RB, GB, BB) = (ar, ag, ab). You can then find the original alpha value with

RW - RB = (1 - a)255

a = 1 - (RW - RB)/255

and then simply divide (RB, GB, BB) by a to recover the components (r, g, b).

The RGB values you read are rounded to integers which will affect the precision with which you can determine the alpha value, but this probably doesn't matter for practical unit tests.

GKFX
  • 1,386
  • 1
  • 11
  • 30