How to get MP4 using JavaCV for live video streaming

Question

I am trying to make an application that gets the streaming video data using JavaCV and send it to Web socket server. Then, the Web socket server distributes the video data to connected client(s). 1. Application gets live streaming data(MP4) from my PC's webcam using JavaCV. 2. Application keeps sending it to web socket server. 3. The server gets it as binary and send it to connected client. 4. Web browser connects to the server and Javascript running on browser shows the live video after receiving it from server.

I am new to JavaCV and OpenCV. The snippet below works for drawing video using com.googlecode.javacv.CanvasFrame with no problem. However, I am not sure how to grab MP4 data as live streaming data.

try {
    FrameGrabber grabber = FrameGrabber.createDefault(0);
    grabber.setFormat("mp4");;
    grabber.setFrameRate(30);
    grabber.setImageWidth(640);
    grabber.setImageHeight(480);
    grabber.start();

    double frameRate = grabber.getFrameRate();
    long wait = (long) (1000 / (frameRate == 0 ? 10 : frameRate));
    ByteBuffer buf = null;

    while (true) {
        Thread.sleep(wait);
        IplImage image = grabber.grab();
        if(image != null) {
            buf = image.getByteBuffer();
            send(buf);  // Send video data using web socket.
        }
    }

} catch (FrameGrabber.Exception ex) {
    Logger.getLogger(WSockClient.class.getName()).log(Level.SEVERE, null, ex);
}

The below are the code for web socket server and Javascript/HTML5 client. The both code works fine if the application reads MP4 file from local disk, and send it to the server. But the data using JavaCV like above seems to be invalid for showing on browser. The server just receives the data and distribute it to the clients. I believe there seems to have no problem. Here is the code.

@OnMessage
public void binaryMessage(ByteBuffer buf, Session client) throws IOException, EncodeException {
    for (Session otherSession : peers) {
        if (!otherSession.equals(client)) { 
            otherSession.getAsyncRemote().sendBinary(buf, new StreamHandler());
        }
    }
}

Here is the Javascript/HTML5.

<body>
<video controls width="640" height="480" autoplay></video><br>
<canvas id="canvas1" width="640" height="480"></canvas><br>   
</body>
<script>
var ws;
var protocol = 'ws';
var host = "localhost:8080";
var url = protocol + "://" + host + "/live/stream";

ws = new WebSocket(url);
ws.binaryType = 'arraybuffer';
ws.addEventListener("open",onOpenWebSocket,false);
ws.addEventListener("close",onCloseWebSocket,false);
ws.addEventListener("message",onMessageWebSocket,false);
window.addEventListener("unload",onUnload,false);
...
function onMessageWebSocket(event){
    var blob = new Blob([event.data], {type: 'video/mp4'});
    var rb = blob.slice(0, blob.size, 'video/mp4');
    video.src = window.URL.createObjectURL(rb);
}
...
</script>

I think I need to obtain MP4 data as stream using JavaCV, but don't know how to do that. Please help. Any comment, suggestions would be appreciated. Thank you.

How to get MP4 using JavaCV for live video streaming

0 Answers0