Live stream HTML5 video draw-to-canvas not working

Question

I use ReactJS to display a live stream (from my webcam) using the HTML5 video element. The OpenVidu media server handles the backend. I would like to use the canvas element to draw the video live stream onto the canvas using the drawImage() method.

I have seen other examples, but in them the video element always has a source. Mine does not have a source - when I inspect the video element all I see is: <video autoplay id="remote-video-zfrstyztfhbojsoc_CAMERA_ZCBRG"/>

This is what I have tried, however the canvas does not work.

export default function Video({ streamManager }) {
  const videoRef = createRef()
  const canvasRef = createRef()

  useEffect(() => {
    if (streamManager && !!videoRef) {

      //OpenVidu media server attaches the live stream to the video element 
      streamManager.addVideoElement(videoRef.current)

      if (canvasRef.current) {
        let ctx = canvasRef.current.getContext('2d')
        ctx.drawImage(videoRef.current, 0, 0)
      }
    }
  })

  return (
    <>
      //video element displays the live stream 
      <video autoPlay={true} ref={videoRef} />

      // canvas element NOT working, nothing can be seen on screen 
      <canvas autoplay={true} ref={canvasRef} width="250" height="250" />
    </>
  )
}

UPDATE: after further investigation I realised I needed to use the setInterval() function and therefore provided the solution below.

user3047450 · Accepted Answer · 2020-01-23T21:22:13.963

The solution is to extract the canvas logic in a self contained component, and use a setInterval() so that it will draw the video element on the canvas, every 100 ms (or as needed).

Video Component

import React, { useEffect, createRef } from 'react'
import Canvas from './Canvas'

export default function Video({ streamManager }) {
  const videoRef = createRef()

  useEffect(() => {
    if (streamManager && !!videoRef) {

      //OpenVidu media server attaches the live stream to the video element 
      streamManager.addVideoElement(videoRef.current)
    }
  })

  return (
    <>
      //video element displays the live stream 
      <video autoPlay={true} ref={videoRef} />

      //extract canvas logic in new component
      <Canvas videoRef={videoRef} />

    </>
  )
}

Canvas Component

import React, { createRef, useEffect } from 'react'

export default function Canvas({ videoRef }) {
  const canvasRef = createRef(null)

  useEffect(() => {
    if (canvasRef.current && videoRef.current) {
      const interval = setInterval(() => {
        const ctx = canvasRef.current.getContext('2d')
        ctx.drawImage(videoRef.current, 0, 0, 250, 188)
      }, 100)
      return () => clearInterval(interval)
    }
  })

  return (
    <canvas ref={canvasRef} width="250" height="188" />
  )
}

score 0 · Answer 2 · answered Jan 23 '20 at 06:34

0

You are using useEffect without parameter which mean your useEffect will call in every render

  useEffect(() => {
     // every render 
  })

If you want to run your drawImage at only mount time then use useEffect with []

  useEffect(() => {
      // run at component mount 
  },[])

Or if you want to run drawImage when any parameter change then pass your parameter to it

   useEffect(() => {
          // run when your streamManager change
      },[streamManager])

use it as per your requirement

answered Jan 23 '20 at 06:34

Prakash Karena

2,525
1
7
16

thanks for the observation, the useEffect was indeed missing the parameter [], but that was not the problem, because in this case the component was not re-rendering itself more than once. After further investigation I found that I just needed a setInterval() function in my component, therefore I've provided the solution below. – user3047450 Jan 23 '20 at 20:26

Live stream HTML5 video draw-to-canvas not working

2 Answers2