2

My app is supposed to take Web Audio streamed from the client, encode it as MP3, before sending it back out to clients via WebSocket.

I can currently encode and pipe to a file like this:

inbound_stream.pipe(encoder).pipe(fs.createWriteStream('audio.mp3'));

And if I already have a file on the server I can do this:

var mp3File = fs.createReadStream('audio.mp3');
          
            mp3File.on('data', function(buffer){
                io.sockets.emit('audio', { buffer: buffer });
            });

However, I want to access the encoded chunks in real time, and send those out to clients - not write to a file.

What I want is this, effectively:

inbound_stream.pipe(encoder).pipe(newReadStream);

        newReadStream.on('data', function(buffer){
            io.sockets.emit('audio', { buffer: buffer });
        });

I've looked at Duplex and Transform streams, but frankly I am still learning and the Prototyping made my head spin.

How do I do this? Thanks.

UPDATE

The solution below from @Nazar Sakharenko certainly does what I wanted, but the overhead of live encoding seems to make this inpossible, so writing the encoded MP3, and pre-buffering it seems to be the only way (thanks to various people for the suggestion.)

However I still have problems with this approach. New question here:

node.js - create a new ReadStream for a new file, when that file reaches a certain size

Community
  • 1
  • 1
user3174541
  • 59
  • 2
  • 9

1 Answers1

1

According to documentation readable.pipe(destination[, options]) the destination should be stream.Writable.

What you can do is to implement your own Writable stream:

const Writable = require('stream').Writable;

var buffer = [];
//in bytes
const CHUNK_SIZE = 102400; //100kb

const myWritable = new Writable({
  write(chunk, encoding, callback) {
    buffer += chunk;
    if(buffer.length >= CHUNK_SIZE) {
       io.sockets.emit('audio', { buffer: buffer});
       buffer = [];
    }

    callback();
  }
});

myWritable.on('finish', () => {
   //emit final part if there is data to emit
   if(buffer.length) {
       io.sockets.emit('audio', { buffer: buffer});
   }
});


inbound_stream.pipe(encoder).pipe(myWritable);

thats all.

Nazar Sakharenko
  • 1,007
  • 6
  • 11
  • Thanks so much for this, I really appreciate it. It worked but unfortunately the encoding jammed up the stream, meaning it can't be used in real time. I wonder if I am better to write the file, and then stream that once it reaches a certain size, using growing-file. – user3174541 Aug 02 '16 at 10:30
  • I'm not familiar with mp3 streaming, probably you can precache some decoded part of mp3 in memory and start streaming after minimal size was decoded. – Nazar Sakharenko Aug 02 '16 at 14:00
  • Thanks @Nazar Sakharenko, I think the pre-cache idea is the only way, but I'm not sure how to correctly stat the file to begin sending chunks after a certain filesize. General workflow is on the client I use the getUserMedia API to stream PCM data from my soundcard to the node JS server, using socket-io-stream. The server encodes it as above to MP3 to save bandwidth and sends it in chunks back to the client, where it's decoded by AudioContext. Ideally I would use PCM data end to end, but I can't get AudioContext to play raw PCM data. – user3174541 Aug 02 '16 at 16:57
  • Just as an addendum to that @Nazar, I can watch the encoded file and use fs.stat to check for a certain size, but the problem is the event keeps firing and interrupts the stream. – user3174541 Aug 02 '16 at 17:03
  • I've created a follow up question that does what you suggested - http://stackoverflow.com/questions/38727384/node-js-create-a-new-readstream-for-a-new-file-when-that-file-reaches-a-certa – user3174541 Aug 02 '16 at 17:56
  • Can you compare buffers when you streaming from file, and from decoder? Something may be corrupted, in buffe, and compare chunk sizes. – Nazar Sakharenko Aug 02 '16 at 18:02
  • Good idea but I went with your other suggestion, pre buffering. And having a recording of the stream is a nice bonus. I just need to know how to keep the stream running once the pre buffer is met. – user3174541 Aug 02 '16 at 18:27
  • I've answered in your other question, same example with saving to file – Nazar Sakharenko Aug 02 '16 at 20:05