Mixing Audio elements into one stream destination for use with MediaRecorder

Question

MediaRecorder only lets you record media of one type per track. So I'm using JQuery to get a list off all audio elements and connect them to the same audio context destination in order to mix all the audio tracks into one audio stream to later be recorded by MediaRecorder. What I have so far works but only captures the first track and none of the others.

Any idea why only one track comes through?

my code:

function gettracks(stream){
    var i = 0;
    var audioTrack ;
    var audioctx = new AudioContext();
    var SourceNode = [];
    var  dest = audioctx.createMediaStreamDestination();
    $('audio').each( function() {

    //the audio element id
    var afid = $(this).attr('id');

    var audio = $('#'+afid)[0];
    console.log('audio id '+ afid+' Audio= '+audio);
    SourceNode[i] = audioctx.createMediaElementSource(audio);

    //dont forget to connect the wires!
    SourceNode[i].connect(audioctx.destination);
    SourceNode[i].connect(dest);


      audioTrack = dest.stream.getAudioTracks()[0];
    stream.addTrack(audioTrack);
    i++;
    });
    }


//from a mousedown event I call
stream = canvas.captureStream();
video.srcObject = stream;

gettracks(stream);
startRecording()
    function startRecording() {
          recorder = new MediaRecorder(stream, {
        mimeType: 'video/webm'
      });
      recorder.start();
    }

What is the `stream` parameter? Can you add the calling code? — Herohtar, Jan 09 '19 at 16:40
Hi, thanks I updated the post will calling code. The video capture works too. but only audio track in it. — Bret, Jan 09 '19 at 16:54
I think the `SourceNode[i].connect(audioctx.destination);` line shouldn't be there. And maybe do `stream.addTrack(dest.stream.getAudioTracks()[0])` after the loop instead of inside it. — Herohtar, Jan 09 '19 at 17:11

score 2 · Answer 1 · answered Jan 15 '19 at 13:39

I would do it like this:

var ac = new AudioContext();
var mediaStreamDestination = new MediaStreamAudioDestinationNode(ac);
document.querySelectorAll("audio").forEach((e) => {
  var mediaElementSource = new MediaElementAudioSourceNode(ac, { mediaElement: e });
  mediaElementSource.connect(mediaStreamDestination);
});
console.log(mediaStreamDestination.stream.getAudioTracks()[0]); // give this to MediaRecorder

Breaking down what the above does:

var ac = new AudioContext();: create an AudioContext, to be able to route audio to something else than the default audio output.
var mediaStreamDestination = new MediaStreamAudioDestinationNode(ac); from this AudioContext, get a special type of DestinationNode, that, instead of send the output of the AudioContext to the audio output device, sends it to a MediaStream that holds a single track of audio.
document.querySelectorAll("audio").forEach((e) => {, get all the <audio> elements, and iterate over them.
var mediaElementSource = new MediaElementAudioSourceNode(ac, { mediaElement: e });, for each of those media element, capture its output and route it to the AudioContext. This gives you an AudioNode.
mediaElementSource.connect(mediaStreamDestination);, connect our AudioNode that has the output of the media element, connect it to our destination that goes to a MediaStream.
mediaStreamDestination.stream.getAudioTracks()[0] get the first audio MediaStreamTrack from this MediaStream. It has only one anyways.

Now, I suppose you can do something like stream.addTrack(mediaStreamDestination.stream.getAudioTracks()[0]), passing in the audio track above.

Thanks for all the detail I really appreciate your help. For some reason I still only get one track coming through. The track that does come through is at a lower volume than it should be. It's almost as if it is being clobbered but there is no trace of any other sounds. This may be a little far fetched but do you know if it is possible to put each track on it's own channel in the stream? — Bret, Jan 16 '19 at 04:31
It's certainly possible. For this, you want a [ChannelMergerNode](https://webaudio.github.io/web-audio-api/#channelmergernode), and then use the [this form](https://webaudio.github.io/web-audio-api/#dom-audionode-connect) of `AudioNode.connect`, changing the last argument to connect to the right input of the `ChannelMergerNode`. — padenot, Jan 21 '19 at 09:16

score 1 · Accepted Answer · answered Jan 24 '19 at 15:32

What if you create a gain node and connect your source nodes to that:

var gain = audioctx.createGain();
gain.connect(dest);

and in the loop

SourceNode[i].connect(gain);

Then your sources flow into a single gain node, which flows to the your destination.

Mixing Audio elements into one stream destination for use with MediaRecorder

2 Answers2