0

I have an Angular project, which uses JSMPpeg library for displaying live-stream video inside canvas. Is there any way to add some listener on frame change, which could detect color on specific part of the canvas? Basically, I need to detect when specific part on the stream video becomes green. I have noticed that there's specific option in JSMpeg:

onVideoDecode(decoder, time) A callback that is called after each decoded and rendered video frame

Probably it should help with detecting frame change, but haven't succeed on implementing it and I still have no ideas about detecting color. Is it even possible?

Thanks

mantebose
  • 93
  • 1
  • 7

1 Answers1

0

If it's a 2d canvas you can get a pixel with

const ctx = someCanvas.getContext('2d');
const imgData = ctx.getImageData(x, y, 1, 1);
const [r, g, b] = imgData.data;

if it's a webgl canvas you can get a pixel with

const gl = someCanvas.getContext('webgl');
const data = new Uint8Array(4);
// in WebGL y = 0 is bottom of canvas
gl.readPixel(x, y, 1, 1, gl.RGBA, gl.UNSIGNED_BYTE, data);
const [r, g, b] = data;

in both cases r, g, and b will be in values from 0 to 255

So you can either check for colors in some rgb range or convert to some other color space

gman
  • 100,619
  • 31
  • 269
  • 393
  • Thanks for response. I have already tried similar way, with click event to get y and x from different parts of canvas, but it always returns  [0, 0, 0, 255] – mantebose Jan 13 '21 at 12:50
  • Should be noted that at least for the 2d API, getting a single pixel through getImageData is almost as slow as getting all the pixels and that it's real slow (buffer is being moved from GPU to CPU at every call...). Better get once the full data and loop through the returned ImageData. – Kaiido Jan 13 '21 at 23:27
  • @mantebose , If the canvas is webgl then you may need to create the webgl context with `preserveDrawingBuffer: true`. If you can't control the creation of the webgl context directly you can augment `getContext`like [this](https://stackoverflow.com/a/55762772/128511) – gman Jan 14 '21 at 01:48