Webaudio Playback from WebSocket has drop-outs

Question

I have a software-defined radio playing an audio stream from a WebSocket server, and a client which consumes the data and plays it using an AudioBufferSourceNode.

It mostly works. The only problem is that there are momentary dropouts every few seconds, presumably caused by the overhead involved in creating each successive AudioBufferSourceNode instance. The WebAudio draft spec says that AudioBuffer should be used for playing sounds that are no longer than a minute or so, and that longer sounds should be played using a MediaElementSourceNode. That doesn't work for me, because I need to play audio from a WebSocket source, and there's no way that I know of to make a media element (e.g. HTML5 audio element) work with a WebSocket.

Maybe I'm trying to do something WebAudio can't support by stringing AudioBufferSourceNode instances together and expecting them to play one after another seamlessly. But it seems there should be a way to play WebSocket data through WebAudio, and indeed auroa.js (together with the aurora-websocket.js plugin) seems to do it. I coded up a client using aurora.js, but I ran into other problems, for which I created an auroa.js issue on Github. In the meantime, I'm hoping that I can do in my client what they seem to have done wrt using WebAudio to play data seamlessly from a WebSocket.

Here is an elided view of my code, to show the implementation I'm using.

var context = ...
var gainNode = ...

var playBuffer = function(buf) {
   var source = context.createBufferSource();
   source.buffer = buf;
   source.connect(gainNode);
   source.start();
};

var socket = ...
socket.binaryType = 'arraybuffer';
socket.addBinaryListener(function (data) {
     context.decodeAudioData(data, playBuffer);
});
socket.connect...

I also tried an implementation wherein I keep track of incoming buffers from the WebSocket and play them in the order received, via an AudioBufferSourceNode, after the 'ended' event is received from the previous AudioBufferSourceNode. This has the same dropout problem that the above implementation has.

Answer 1

Your stream is really guaranteed to get complete audio files in each network chunk? (decodeAudioData does not work with partial MP3 chunks.)

It seems like (from the code snippet above) you're just relying on network timing to get the stream chunks started at the right time? That's guaranteed not to line up properly; you need to keep a bit of latency in the stream (to handle inconsistent network), and carefully schedule each chunk. (The bit above that makes me cringe is source.start() - with no time param that will keep the chunks scheduled one right after another. i.e.:

var nextStartTime = 0;

function addChunkToQueue( buffer ) {
    if (!nextStartTime) {
        // we've not yet started the queue - just queue this up,
        // leaving a "latency gap" so we're not desperately trying
        // to keep up.  Note if the network is slow, this is going
        // to fail.  Latency gap here is 1 second.
        nextStartTime = audioContext.currentTime + 1; 
    }
    var bsn = audioContext.createBufferSource();
    bsn.buffer = buffer;
    bsn.connect( audioContext.destination );
    bsn.start( nextStartTime );

    // Ensure the next chunk will start at the right time
    nextStartTime += buffer.duration;
}

In addition, depending on how big your chunks are, I'd wonder if garbage collection isn't contributing to the problem. You should check it out in the profiler.

The onended path is not going to work well; it's reliant on JS event handling, and only fires AFTER the audio system is done playing; so there will ALWAYS be a gap.

Finally - this is not going to work well if the sound stream does not match the default audio device's sample rate; there are always going to be clicks, because decodeAudioData will resample to the device rate, which will not have a perfect duration. It will work, but there will likely be artifacts like clicks at the boundaries of chunks. You need a feature that's not yet spec'ed or implemented - selectable AudioContext sample rates - in order to fix this.