Use Alpha-Channeled video to mask HTML content

Question

I'm trying to achieve effect that is currently possible in after effect, and the effect is about clip-masking (luma matte) one video with another video so the final out-put would be a transparant video. But I'm trying to make a dynamically transparent HTML content.

Video would look something like this: https://www.youtube.com/watch?v=jfzcrO4694E

Basically revealing HTML content with changing transparency.

This has to be something about + , but after googling around an hour, could't find anything that would describe and effect of black\white video that would change HTML content transparency.

Any ideas?

Answer 1

HTML5 does not support alpha channeled video (such as MOV 32-bit etc.).

The only option is to use matte composited like your video combined with a canvas element. However, canvas only composite already existing alpha channel, and as a matte is a solid layer and not an alpha, you have to convert it to alpha.

This means you have to draw and iterate each frame to a canvas, and convert the grey scale to transparency levels. However, this is very CPU intensive and will affect the computer performance/resources. The video must also fulfill CORS requirement in order to not taint the canvas (in which case you can't extract the pixels).

Here is how

For each frame:

• grab a level from any channel for each pixel, we'll use red here
• subtract the value from 255 since matte white here will be the target (reversed, will reveal)
• clear image data which is the matte itself or you will get the white matte leaking into the data
• set the calculated value as new alpha channel value

Example loop (assuming video is created dynamically, has its source(s) set and has triggered event for starting - canvas is placed where you want to mask and context ctx is initialized):

ctx.globalCompositeOperation = "copy";       // will clear previous alpha

function unmask() {
    ctx.drawImage(video, 0, 0);              // draw current video frame

    var idata = ctx.getImageData(0, 0, ctx.canvas.width, ctx.canvas.height),
        data = idata.data,
        len = data.length, i = 0;            // ..image data from canvas

    while(i < len) {
        var alpha = 255 - data[i];           // in this matte, white = fully transparent
        data[i] = data[i+1] = data[i+2] = 0; // clear matte to black
        data[i+3] = alpha;                   // set alpha
        i += 4;                              // next pixel
    }

    ctx.putImageData(idata, 0, 0);           // update canvas
    requestAnimationFrame(unmask);           // todo: add criteria to stop this
}

Speed-up tip: use blacks (or pre-invert the matte video), then use Uint32Array view on the data, right-shift a pixel long-word (<<24) back in place.

---

There is a slight possibility that you will be able to do this with the new CSS custom filters (aka CSS shaders), but this is not widely supported at the time of this writing. It will also require the use of shader language (GLSL). But I found it worth mentioning at least.

Answer 2

The other answers so far assume a 2D canvas context. You can do this much more efficiently using WebGL. This way, all the real work is done in the GPU.

You need a fragment shader that does basically what the loop does in Ken Fyrstenberg's answer: find the luminance of each pixel, subtract it from the maximum value (one, in GLSL, rather than 255), and use that as the alpha value. I'm going to follow his lead in (a) using only the green channel as the luminance, and (b) setting the output color channels to black. Here's how I'd do it:

precision lowp float;
uniform sampler2D sampler0;
varying vec2 v_texCoord;
void main(void) {
  vec4 color = texture2D(sampler0, v_texCoord);
  float alpha = 1.0 - color.g;
  gl_FragColor = vec4(0.0, 0.0, 0.0, alpha);
}

For each frame, then, you render a rectangle to the WebGL canvas, with your video as the texture source, and the above fragment shader. I've started writing up an example of how to do this, but it's going to take some work. How much interest is there? Let me know and I'll try to finish it up if there's enough demand.

EDIT: You can probably avoid having to learn too much about WebGL by using glfx.js. I haven't looked very deeply at it, but I assume it can be made to work with a video as the image source.

Answer 3

The white area of your example video is increasingly revealed by the receding black area.

You can use your existing video + canvas to reveal underlying html content.

Create a video element containing the video you already have made.

Use a timer to grab frames of your playing video. requestAnimationFrame is highly suggested because it synchronizes with your display refreshes.

In each animation loop, draw a frame of your video on the canvas. You can draw your video on the canvas because the canvas because the canvas can use a video element as its image source.

Once a frame of your black-mist is on the canvas, you can make any white areas of the canvas transparent by:

• getting the color data of every pixel on the canvas using context.getImageData

• if the color of any pixel is "white-ish" you can change the alpha of that pixel to 0 (meaning transparent).

• put the modified pixel data back onto the canvas with context.putImageData

So overall:

• Hide your playing video element

• Use CSS to position your canvas element over the html content you wish to reveal.

• Create an animation loop that draws a video frame on the canvas and converts the white-ish pixels to transparent.

• The html content will be revealed by the canvas playing frames of your video with the white-ish pixels knocked into full transparency.