2

I've written a sample in nodejs which streams some input to the client via websocket connection in mp4 format. On the client side, the mp4 packages are added to a MediaSourceBuffer.

This runs fine, but only if the client gets the stream from the beginning with the first package. So another client can't play the current Stream, because he won't get the Stream from the beginning.

I tried (try&error) to save the first package ffmpeg sends and send this at the beginning of a new connection, then the current stream. Then the MediaSourceBuffer breaks because of encoding error..

Here is the ffmpeg command :

-i someInput -g 59 
-vcodec libx264 -profile:v baseline 
-f mp4 -movflags empty_moov+omit_tfhd_offset+frag_keyframe+default_base_moof
-reset_timestamps 1
-

The part "empty_moov+omit_tfhd_offset+frag_keyframe+default_base_moof" should make the Streampackages independent in putting the moovatom at the beginning of each part and sizing the parts in 59 frames each by keyframe, so I don't get it why I can't view the Stream beginning after the start.

Jason Aller
  • 3,541
  • 28
  • 38
  • 38
curtiss
  • 181
  • 1
  • 2
  • 9

1 Answers1

3

The output of that command is not a 'stream' per se. It is series of concatenated fragments. Each fragments must be received in its entirety. If a partial fragment is received it will confuse the parser to the point where it can not identify the start of the next fragment. In addition, the first fragment output is called an initialization fragment. This initialization fragment must be sent to the client first. After that any fragment can be played. Hence it must be cached by the server.

szatmary
  • 29,969
  • 8
  • 44
  • 57
  • I tried to send the first package given from ffmpeg, without success. I chached the first sended package and on a new client connect, i sended the chached package it to the new socket. After sending the first package, i startet sending the current stream. - how could i cache the "initial package" ? – curtiss Aug 10 '15 at 20:07
  • What exactly do you mean by "initial package"? The "initialization fragment" will likely consist of an `ftyp` and a `moov` box. But there could be others. – szatmary Aug 10 '15 at 20:35
  • I mean the first output given from child_process.spawn call of ffmpeg `code` ffmpeg.stdout.on("data",function(data) { if(initialPackage == null){ initialPackage = data; } console.log("Sending to "+clients.length); clients.forEach(function(client) { client.send(data, {binary: true, mask: false}); }); }); – curtiss Aug 12 '15 at 09:52
  • and with initial package i mean the initialization fragment ;) – curtiss Aug 12 '15 at 11:40
  • How do you determine the length of the initialization fragment? – szatmary Aug 12 '15 at 15:06
  • I supposed the first output from ffmpeg would be the initial fragment, i dont validate it. I don't even know how to do that... – curtiss Aug 12 '15 at 19:52
  • I know exactly how to solve this. But I can't do the work for you. You need to send an initialization fragment, followed by complete video/audio fragments. You'll need to do some research. – szatmary Aug 13 '15 at 18:08
  • ;) wasnt meant so. It seems like you could help here in this situation. I dont ask for code or complete solutions! Im thankful for your help! So far, my searches doesnt show me usefull results. You said on another page / question on Stackoverflow, that this cant be done with ffmpeg? – curtiss Aug 14 '15 at 10:56
  • someone already did the work for you https://www.npmjs.com/package/stream-transcoder – mkoryak Sep 29 '15 at 19:42
  • Thats not right. It returns one stream and doesnt handle multiple clients – curtiss Nov 05 '15 at 17:09
  • szatmary, you are right! I meant mkoryak's comment above. – curtiss Nov 05 '15 at 18:34
  • 1
    As @szatmary said, you'll need to do a bit of research. Making an `mp4` parser it's relatively easy, and something you'll need to do if you want to use MSE. You can start by looking at http://xhelmboyx.tripod.com/formats/mp4-layout.txt where you'll find a description of the boxes you need. You also can google ISO BMFF which is the standard that defines `mp4` files content. – Pablo Montilla Nov 05 '15 at 19:48