Can I capture stream from a hidden canvas in javascript?

Question

I'm currently developing an API which takes a videoTrack as input and return a processed videoTrack. I've managed to draw the processed video on a canvas and hoping use the canvas.captureStream() to capture the videoStream from it.

As it turns out that I can capture a non-blank stream only if I load the canvas into the DOM document.body.appendChild(myTempCanvas) and keep it displayed, the stream turns blank if I hide the canvas myTempCanvas.style.display="none".

So is there any way to capture the stream and keep the canvas hidden as well?

Example: If uncomment the line canvas.style.display = "none";, then the output_video turns blank as well.

<!DOCTYPE html>
<html lang="en">
<head>
  <title>Demo</title>
</head>
<body>
<div class="container">
  <video autoplay class="input_video" ></video>
  <canvas class="output_canvas"></canvas>
  <video autoplay class="output_video" ></video>
</div>
<script>
    const video = document.getElementsByClassName("input_video")[0];
    const canvas = document.getElementsByClassName("output_canvas")[0];
    const output_video = document.getElementsByClassName("output_video")[0];
    let imageCapture = null;
    let tmpStream = null;

    navigator.mediaDevices.getUserMedia({
      video: true
    })
    .then((stream) => {
      imageCapture = new ImageCapture(stream.getVideoTracks()[0]);
      window.requestAnimationFrame(frameloop);
      tmpStream = canvas.captureStream(20);
      // canvas.style.display = "none";
      output_video.srcObject = tmpStream;
    })

    function frameloop() {
      imageCapture.grabFrame()
      .then(imageBitmap => {
        drawCanvas(canvas, imageBitmap);
        window.requestAnimationFrame(frameloop);
      })
    }

    /* Utils */
    function drawCanvas(canvas, img) {
      canvas.width = getComputedStyle(canvas).width.split('px')[0];
      canvas.height = getComputedStyle(canvas).height.split('px')[0];
      let ratio  = Math.min(canvas.width / img.width, canvas.height / img.height);
      let x = (canvas.width - img.width * ratio) / 2;
      let y = (canvas.height - img.height * ratio) / 2;
      canvas.getContext('2d').clearRect(0, 0, canvas.width, canvas.height);
      canvas.getContext('2d').drawImage(img, 0, 0, img.width, img.height,
              x, y, img.width * ratio, img.height * ratio);
    }

</script>
</body>
</html>


</script>
</body>
</html>

Did you tried to hidde your canvas in a div with `height:0;` `whidth:0;` `overflow: hidden;` and `position:fixed;` ? Something like this should do the trick. — Jordan Breton, Sep 24 '21 at 08:01
None of the previous duplicate targets were about this issue, so I did reopen the question. However, what you describe as not working should *"just work"*, so please provide a [mcve] and info about your environment. — Kaiido, Sep 24 '21 at 08:59
@PeterO. I do it everytime I can provide a full answer. I suggested something to help the OP getting something work, but since I can't provide a precise explanation of what happend there (I have only some clues), I prefer letting someone else provide a proper documented answer on the **why** part of this, without posting a useless/incomplete and/or wrong answer. — Jordan Breton, Sep 24 '21 at 11:18
Once again, please provide a [mcve]. To answer the question in the title, [yes you can](https://jsfiddle.net/x9dhg3sy/). That you need the canvas visible means that you are doing something at least *odd*. There are restrictions with non-visible muted videos that are in the DOM, but none (to my knowledge) about canvas. There are limitations with many timing methods when the tab is in background, but setting the style to display none should have no influence on it. So, yes, we need a minimal repro to be able to help you at the best. The hack that was provided earlier is a hack, not a solution. — Kaiido, Sep 25 '21 at 01:53
@Kaiido: Thanks for the advice, I just added one. I'm using a loop of requestAnimationFrame to draw on a canvas. The original code uses webGL2 to process the frame while here I just use grabFrame for a simpler code. The code is only able to get a non-blank stream when the canvas is not hidden. — 古今中, Sep 26 '21 at 08:43

score 3 · Accepted Answer · answered Sep 26 '21 at 10:54

In your drawCanvas function you are setting the canvas width and height to its computed CSS width and height.
When you hide the canvas through display:none, or when it's not appended in the document, these values will be "0px", because the canvas is not in the CSSOM.

A canvas with zero width or height can't produce any output.

Setting the canvas width or height is a very slow operation, in Chrome, this will allocate a full new image bitmap every time you do so, added to the resetting of all the canvas context's default values.
So never set the canvas width or height in all the frames, only when it actually did change, and don't set it to the computed size of your <canvas> element, set it to the size of your input image, and then set the CSS size accordingly.

Can I capture stream from a hidden canvas in javascript?

1 Answers1