Why does this decrease the audio quality?

Question

I am streaming audio in my flask app from a client to the server, but the received audio has very low quality.

On the client, I preprocess the audio buffer as follows:

this.node.onaudioprocess = function(e){
      var buf = e.inputBuffer.getChannelData(0);
      var out = new Int16Array(buf.length);
      for (var i = 0; i < buf.length; i++){
        var s = Math.max(-1, Math.min(1, buf[i]));
        out[i] = s < 0 ? s * 0x8000 : s * 0x7FFF;
      }
      socket.emit('audio event',{data: out})
      return;
}

On the server side, I receive the audio as follows:

audio_file = open('tempfile.raw', 'w')

@socketio.on('audio event')
def audio_message(message):
    dat = [v[1] for v in sorted(message['data'].iteritems())]
    n = len(dat)
    byteval = struct.pack('<'+str(n)+'h',*dat)
    audio_file.write(byteval)

But the resulting audio sounds metallic, interrupted and noisy. Here's how the resulting waveform looks:

Where in my code is the audio quality lost? How can I stream audio without quality loss?

score 2 · Accepted Answer · answered Sep 02 '16 at 21:57

My first impression of how you process the audio is that it is too slow to work in real time.

On the client, you iterate through every single sample, apply bounds checking (do you really need to do that?) then convert from a float32 to a int16 format applying a conditional and a multiplication to each individual sample.

Then on the server side you do yet another loop through each sample, only to get the samples into a list (isn't the data already coming to you as a list?). And only then then you pack that list into a binary array that gets written to disk.

This is a lot of work to just write a buffer, you are probably losing data.

Here is what I recommend that you try: remove all conversions and see if you can get the data flowing through the system in the native float32 format. With socket.io you can send the float32 data packed directly from the client. Haven't tested this, but I believe socket.emit('audio event',{data: buf.buffer}) will get the binary payload sent out directly and with no client-side conversion. Then on the server, message['data'] will be a binary payload that you can write directly to disk. To check if the data looks good, you can use audacity, using the 32-bit float option in the Import Raw dialog.

Once you get the raw float32 data working, if you need the data in another format, you can see if adding a conversion (hopefully in only one place) still allows you to sustain real time. I suspect you may need to code this conversion in C/C++, as Python is too slow for this type of thing. If you are going down this path, looking into Cython might be a good idea.

What kind of conversion happens in buf.buffer? Does that convert float32array to a binary buffer? — user2212461, Sep 04 '16 at 14:36
No, the `buffer` property returns the underlying binary structure of the array, without any conversions. See the documentation for JavaScript typed arrays: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Typed_arrays. — Miguel Grinberg, Sep 04 '16 at 16:17

Why does this decrease the audio quality?

1 Answers1