how to getUserMedia and record the video mixing the mp3 with javascript? mp3 can play pause and stop

Question

I am using getUserMedia and mediaRecorder API to record an video from webcam.

I am using chrome version 80.

How to getUserMedia and record the video mixing the mp3 with javascript? mp3 can play pause and stop

I don't know how to mixing the mp3 to the video stream on live.

When I removeTrack and addTrack, I stop on MediaRecorder fail.

show Error: Failed to execute 'stop' on 'MediaRecorder': The MediaRecorder's state is 'inactive'.

my code on codepen: https://codepen.io/zhishaofei3/pen/eYNrYGj

and prime codes:

function getFileBuffer(filepath) {
  return fetch(filepath, {method: 'GET'}).then(response => response.arrayBuffer())
}

function mp3play() {
  getFileBuffer('song.mp3')
  .then(buffer => context.decodeAudioData(buffer))
  .then(buffer => {
    console.log(buffer)
    const source = context.createBufferSource()
    source.buffer = buffer
    let volume = context.createGain()
    volume.gain.value = 1
    source.connect(volume)
    dest = context.createMediaStreamDestination()
    volume.connect(dest)
    // volume.connect(context.destination)
    source.start(0)

    const _audioTrack = stream.getAudioTracks();
    if (_audioTrack.length > 0) {
      _audioTrack[0].stop();
      stream.removeTrack(_audioTrack[0]);
    }

    console.log(dest.stream)
    console.log(dest.stream.getAudioTracks()[0])
    stream.addTrack(dest.stream.getAudioTracks()[0])
  })
}

thank you !

Answer 1

Many containers don't support adding/removing tracks like that, and it's doubtful the Media Recorder API does at all. It's an unusual thing to do.

You need to create the stream you're going to record before instantiating Media Recorder, with all of the tracks you want. Therefore, you need to do things in this order:

1. Set up your AudioContext.
2. Call getUserMedia(). (And while you're at it, set audio: false in your constraints. No need to open a microphone if you're not using one.)
3. videoStream.getVideoTracks() and dest.stream.getAudioTracks() to get all of the tracks.
4. Create a new MediaStream with those tracks. new MediaStream([audioTrack, videoTrack])

Now, run your MediaRecorder on this new MediaStream and you'll have what you want.