Node.js - MJPEG TCP stream to base64 images

Question

Based on paparazzo.js lib, I'm trying to get base64 images from a MJPEG stream (streamed over TCP with GStreamer) in a Node.js server, and to send them to the clients via websockets.

I think I'm pretty close, but my images are corrupted. Here is the code I'm using :

var boundary = "----videoboundary";
var data = "";

var tcpServer = net.createServer(function (socket) {

    socket.on('data', function(chunk) {

        var boundaryIndex = chunk.toString().indexOf(boundary);

        if (boundaryIndex !== -1) {

            // Get the image's last piece of data :
            data += chunk.toString().substring(0, boundaryIndex);

            // Convert the data to a base64 image and broadcast it :
            var image = new Buffer(data).toString('base64');
            io.sockets.emit('image', image);

            // Reset the data :
            data = '';

            // Get the remaining data (with new image's headers) :
            var remaining = chunk.toString().substring(boundaryIndex);

            // Remove the new image's headers and add the remaining data :
            var contentTypeMatches   = remaining.match(/Content-Type:\s+image\/jpeg\s+/);
            var contentLengthMatches = remaining.match(/Content-Length:\s+(\d+)\s+/);

            if(contentLengthMatches != null && contentLengthMatches.length > 1) {

                var newImageBeginning = remaining.indexOf(contentLengthMatches[0]) + contentLengthMatches[0].length;
                data += remaining.substring(newImageBeginning);
            }
            else if(contentTypeMatches != null) {

                var newImageBeginning = remaining.indexOf(contentTypeMatches[0]) + contentTypeMatches[0].length;
                data += remaining.substring(newImageBeginning);
            }
            else {

                var newImageBeginning = boundaryIndex + boundary.length;
                data += remaining.substring(newImageBeginning);

                io.sockets.emit('error', { message: 'Could not find beginning of next image' });
            }
        }
        else {
            data += chunk;
        }
    });
});

Any idea ?

Thanks

I don't know for certain, but I believe the issue is how you're using `toString()`... by default, it treats buffers as binary encoded, you may need to specify base64: `toString('base64')`, give that a shot — Jason, Apr 29 '14 at 17:08
Yes the problem is probably caused by the toString(), but I need it to detect the boundary and headers. Maybe I need to do something like : chunk -> toString() -> get headers -> encode headers in binary -> slice original chunk. — Tim Autin, Apr 30 '14 at 15:08
You said you were trying to "get base64 images" which I interpreted to mean that the images were base64 encoded before you received them, is that not the case? — Jason, Apr 30 '14 at 15:59
No, I'm receiving a binary MJPEG stream from GStreamer via TCP, and I would like to convert it in a sequence of base64 images. Then I will send these images to the client via websockets. — Tim Autin, Apr 30 '14 at 16:12
Ah. Well, then you're probably gonna run into issues, as mscdex states, by trying to work with binary data as a string. If you want to handle it directly (rather than through a module that takes care of the details for you) then you'll likely have to loop through the buffer byte by byte to search for your boundary, or you may be able to get it to work by re-encoding it with `toString` to a separate variable that you just inspect for the location of your boundary, and use that location to deal with the buffer directly. — Jason, Apr 30 '14 at 16:49
Or, I should say, you probably won't be able to use the location directly, but rather a way to get at the number of bytes in at which the boundary occurs. — Jason, Apr 30 '14 at 16:56

score 1 · Accepted Answer · answered Apr 29 '14 at 17:10

chunk.toString() converts the binary Buffer to a utf8-encoded string (by default), so for binary image data that will probably cause you some problems.

Another option that might help simplify things for you is to use the dicer module. With that, your code may look like:

var Dicer = require('dicer');
var boundary = '----videoboundary';

var tcpServer = net.createServer(function(socket) {
    var dice = new Dicer({ boundary: boundary });

    dice.on('part', function(part) {
      var frameEncoded = '';
      part.setEncoding('base64');
      part.on('header', function(header) {
        // here you can verify content-type, content-length, or any other header
        // values if you need to
      }).on('data', function(data) {
        frameEncoded += data;
      }).on('end', function() {
        io.sockets.emit('image', frameEncoded);
      });
    }).on('finish', function() {
      console.log('End of parts');
    });
    socket.pipe(dice);
});

OK for the toString, it seems logical, but do you know how can I detect the boundary and headers in the binary chunk ? I tried the code you gave with Dicer, it never outputs anything, any hint ? — Tim Autin, Apr 30 '14 at 15:14
Got it working, it was just my boundary that was wrong. Many thanks ! — Tim Autin, May 12 '14 at 08:37
(I put here the complete workflow : http://stackoverflow.com/questions/23359736/solutions-to-stream-from-a-decklink-card-to-browsers-gstreamer-tcp-mjpeg/23605892#23605892) — Tim Autin, May 12 '14 at 13:37

Node.js - MJPEG TCP stream to base64 images

1 Answers1

Linked