3

I have a MediaStream from a WebRTC remote peer from which I would like to create a video recording in the browser.

I'm currently creating the MediaRecorder like this:

const recorder = new MediaRecorder(mediaStream);

The original stream has a frame rate of e.g. 30fps. I would like the recording to have a lower framerate, e.g. 12fps.

So far the only strategy I can find is to create an intermediary canvas, repeatedly copy the original stream to the canvas, and create a new stream with the desired framerate based on the canvas, something like this:

const video = document.getElementById('my_video_element');
// ... do more to set up video here
video.srcObject = mediaStream;
const canvas = document.createElement('canvas');
canvas.width = 1280;
canvas.height = 720;
setInterval(() => {
  canvas.getContext('2d').drawImage(video, 0, 0, canvas.width, canvas.height);
}, 18);
const desiredFps = 12;
const recorder = new MediaRecorder(canvas.captureStream(desiredFps));

Is there a better/simpler way?

Note: I am aware that the videoBitsPerSecond setting can be used to reduce the bitrate of the output recording. But I find that even for a fixed videoBitsPerSecond setting value, reducing the fps of the canvas stream reduces the effective bitrate of the MediaRecorder.

fblundun
  • 987
  • 7
  • 19
  • 1
    you might want to ask on https://github.com/w3c/mediacapture-record -- this would be a sensible thing. Implementations might be more difficult if they have to drop frames to achieve a desired target framerate though. – Philipp Hancke Jan 19 '22 at 15:00

1 Answers1

5

You have the right approach. You're doing temporal resampling. That's what ffmpeg would do if you did it with native code.

If you're starting with 30fps, you should resample to either 10fps or 15fps. That way you can keep every other decoded frame, or every third frame. That makes for a nicer-looking recording.

Resampling from 30fps to 12fps and getting tolerable output is possible, but it's a computational monster because of the need for temporal interpolation. Relying on browser WebRTC to do that? Not wise.

O. Jones
  • 103,626
  • 17
  • 118
  • 172
  • so what is the alternative you are proposing? – Philipp Hancke Jan 19 '22 at 23:04
  • I'm not proposing any alternative. The original questioner's approach: frame-dropped video to canvas to MediaRecorder is the way to go. Unless it's feasible to use offline ffmpeg to reduce the recorded bitrates. But that's not a browser-based solution, not even close. – O. Jones Jan 19 '22 at 23:07
  • Thanks for the answer. To check - are you saying that the problem with using my `canvas.captureStream(12)` approach with a target framerate of 12fps is that some output frames would correspond to 2 input frames and some to 3 input frames, making the output look worse than if every output frame corresponded to the same number of input frames? – fblundun Jan 20 '22 at 00:28
  • 1
    Exactly ... resampling looks better if you do it n - for - 1 . – O. Jones Jan 20 '22 at 00:39