8

Edit: I don't necessarily need a solution to this problem--rather I'd like to understand why it's occurring. I don't see why I should be getting the odd results below...

Although this question is directed towards an issue I'm having with an HTML5 canvas application, I think the problem is less specific.

I have an HTML5 canvas app that allows you to stamp images on the screen. These images are 32bit PNG's, so I'm working with transparency. If I stamp a highly transparent image in the same location many times (roughly 100), I end up with an absolutely terrible result:

https://i.stack.imgur.com/K0YYd.png

The color of the image that I'm using as a stamp is RGB(167, 22, 22) and the background that I'm stamping onto is RGB(255, 255, 255). Here's the source image, if anyone's interested:

https://i.stack.imgur.com/Wm9Dx.png

As you can tell, the image has extremely low alpha levels. Likely about 2/255 to 5/255 or so. What I would expect to happen is that if you repeatedly apply the image stamp to the canvas enough times, you'll get pixels of color RGBA(167, 22, 22, 255). Unfortunately, I'm getting a mixed bag of colors including some very odd regions of gray with a value of RGB(155, 155, 155).

I just loaded up Excel and plugged in the equation for source-over alpha blending (Wikipedia reference) and I seem to be converging to RGB(167, 22, 22) after enough iterations. I'm probably missing something fundamental about alpha blending operations and how the HTML5 canvas implements source-over compositing... can anyone help straighten me out?

Thanks!

Note: this question is similar to my issue, but I don't quite understand why I'm getting the results I've posted here.

Community
  • 1
  • 1
Xenethyl
  • 3,179
  • 21
  • 31
  • Have you tried the accepted answer of the question you stated is close to your problem? It would seem that solution may work for you also. – James Black Mar 20 '11 at 00:06
  • @James Black I have not. I don't really understand how there is a lack of precision, which in turn is keeping me from seeing how to fix the problem. If I use the source-over alpha blending equation, eventually I'll get the color I want--but that's not what's happening with the HTML5 canvas...? – Xenethyl Mar 20 '11 at 00:08
  • Can you post another link to the image? It is no longer available from imgur – sinelaw Oct 06 '11 at 08:23
  • @sinelaw Sure, I just re-uploaded it for you. – Xenethyl Oct 06 '11 at 08:55
  • Thanks. BTW, what did you end up doing? – sinelaw Oct 06 '11 at 08:57
  • @sinelaw I actually ended up writing a WebGL codepath to circumvent this issue, although that didn't seem to avoid it 100% of the time. I've had the project on the back burner for a while, but this gives me a good reason to play around with it again. I'm pretty sure Nathan Ostgard identified a solution. I'll play with it today and see if I can come up with a generalized, clean fix for it. If I recall correctly Firefox and Chrome had varying results, so it might take a bit of experimentation. I'll report back. :) – Xenethyl Oct 06 '11 at 09:05
  • @sinelaw After spending some time working on this today, I'm pretty sure the only way to completely eliminate the issue is to blend the colors manually via `getImageData()` and a homespun blend function. Firefox and IE on Windows produce different results from Chrome on Windows and from every browser on Mac (browsers on Mac are consistent). WebGL helps here because you can use pixel shaders, but if you're using a `2d` canvas and absolutely need perfect blending, you'll have to do it yourself as far as I can tell. – Xenethyl Oct 07 '11 at 05:01

1 Answers1

7

The precision and rounding rules of canvas math internals are mostly undefined, so it's hard to say exactly what's happening here. All we really know is that the pixels are unsigned bytes, and the alpha is premultiplied.

However, we can get some information by using getImageData to inspect the pixels as the stamp is drawn, like so:

var px = 75;
var py = 100;
var stamp = new Image;
stamp.onload = function() {
  for (var i = 0; i < 100; ++i) {
    imageData = context.getImageData(px, py, 1, 1);
    console.log(Array.prototype.slice.call(imageData.data, 0, 4));
    context.drawImage(stamp, 0, 0);
  }
};
stamp.src = 'stamp.png';

The sample at px = 75, py = 100 is right in the middle of a gray blob. After drawing the stamp once on a white canvas, the log reads:

[254, 254, 254, 255]

At px = 120, py = 150, the sample is in the middle of a red area. After drawing the stamp once, the log reads:

[254, 253, 253, 255]

So, it looks like the canvas was modified by (-1, -1, -1) for the grey pixel, and (-1, -2, -2) for the red pixel.

Sampling these same pixels in the stamp image using RMagick gives:

[167, 22, 22, 1]  // x = 75, y = 100
[167, 22, 22, 2]  // x = 120, y = 150

Working through the math, using the standard alpha blending equation, you can test each of the color values:

function blend(dst, src) {
  var a = src[3] / 255.0
  return [
    (1.0 - a) * dst[0] + a * src[0],
    (1.0 - a) * dst[1] + a * src[1],
    (1.0 - a) * dst[2] + a * src[2]
  ];
}

console.log(blend([255, 255, 255], [167, 22, 22, 1]));
// output: [254.6549..., 254.0862..., 254.0862...]

console.log(blend([255, 255, 255], [167, 22, 22, 2]));
// output: [254.3098..., 253.1725..., 253.1725...]

From this, we can guess that the canvas blending code is actually flooring the results, instead of rounding them. This would give you a result of [254, 254, 254] and [254, 253, 253], like we saw from canvas. They're likely not doing any rounding at all, and it's being floored implicitly when cast back to an unsigned byte.

This is why the other post recommends storing the image data as an array of floats, doing the math yourself, and then updating the canvas with the result. You get more precision that way, and can control things like rounding.

Edit: In fact, this blend() function isn't exactly right, even when the results are floored, as the canvas pixel values for 120, 150 stabilize at [127, 0, 0], and this function stabilizes at [167, 22, 22]. Similarly, when I drew the image just once into a transparent canvas, getImageData on the pixel at 120, 150 was [127, 0, 0, 2]. What?!

It turns out that this is caused by premultiplication, which seems to be applied to loaded Image elements. See this jsFiddle for an example.

Premultiplied pixels are stored as:

// r, g, b are 0 to 255
// a is 0 to 1
// dst is all 0 to 255
dst.r = Math.floor(r * a);
dst.g = Math.floor(g * a);
dst.b = Math.floor(b * a);
dst.a = a * 255;

They are unpacked later as:

inv = 1.0 / (a / 255);
r = Math.floor(dst.r * inv);
g = Math.floor(dst.g * inv);
b = Math.floor(dst.b * inv);

Running this pack/unpack against [167, 22, 22, 2] reveals:

a = 2 / 255;                                // 0.00784
inv = 1.0 / (2 / 255);                      // 127.5
r = Math.floor(Math.floor(167 * a) * inv);  // 127
g = Math.floor(Math.floor(22 * a) * inv);   // 0
b = Math.floor(Math.floor(22 * a) * inv);   // 0
Nathan Ostgard
  • 8,258
  • 2
  • 27
  • 19
  • 1
    Thank you very, very much for your thorough reply. I understand what you are saying here and now see what the other question was referring to with the loss of precision. Unfortunately, I don't think that's my issue here. If we continue to follow the process you outlined here, won't we eventually get [167, 22, 22, 255] for (120, 150)? The destination image starts at [255, 255, 255, 0] and yet somehow I'm ending up with a non-solid red color and an odd shade of gray as a result of the alpha blending equation used by the browser... – Xenethyl Mar 22 '11 at 03:23
  • @Xenethyl Good point, I didn't notice that. I did some more digging and it looks like both dst and src are being floored before adding. I've updated the answer with an example. – Nathan Ostgard Mar 22 '11 at 04:31
  • 1
    @Xenethyl Gah! After trying to repro this in a jsFiddle, with normal color fills, I was seeing even more weirdness. I think I've narrowed this down to the premultiply... – Nathan Ostgard Mar 22 '11 at 06:16
  • Thanks again for your very involved response. I actually came across premultiplied alpha values while I was trying to research this problem but didn't think anything of it at the time. I haven't had the time yet to try and play with the stamp image to revert the premultiplication, but I think based on the results you came up with it is indeed the culprit. I really appreciate your time here--this would have driven me crazy for quite a while. :) – Xenethyl Mar 23 '11 at 08:10