4

I'm trying to get h.264 NAL units from a MOV file on the iPhone, in order to RTP h.264 video from the iPhone camera to a server.

Apple's API does not allow direct access to the encoded bitstream from the camera output, so I can only access the MOV file, while it's being written.

I've parsed the MOV file into Atoms, according to Apple's MOV structure reference but now i need to extract the NAL units from the mdat atom in order to pack it to RTP and stream it.

I'd be glad for some help here because i can't find documentation about the mdat structure. Thanks!

Avishay Cohen
  • 2,418
  • 1
  • 22
  • 40
  • Steve McFarlin ( http://stackoverflow.com/users/227021/steve-mcfarlin ) has such a library. – magma Aug 04 '12 at 01:58
  • @avishic How have you solved this finally? Do you use any library or parse frames by hand? Could you please post answer for your question here? Thanks. – some.birdie Feb 11 '13 at 09:02
  • 1
    @Anastasia Haven't solved it completely. It seems that parsing the mdat is very complex if it contains more than a single video channel. But, if it does contain a single video channel then, the mdat is simply a collection of NAL units. So according to their headers (mdat & nal units) you can know how many bytes in each nal unit and parse it yourself. Not ideal, but a working solution. – Avishay Cohen Feb 11 '13 at 13:10
  • @avishic So, now you can get H.264 frames from hardware encoder, right? Could you share your code? Please contact me (see my profile). – some.birdie Feb 14 '13 at 07:59
  • @Anastasia Sorry, i can't share the code because it has legal rights. But check out the link to Apple's documentation about atoms & mdat in the question above. – Avishay Cohen Feb 14 '13 at 14:28
  • @avishic SPS/PPS are not written until the file is complete. So how do you get them? Do you use "streaming" quicktime file setting movieFragmentInterval property, so that SPS+PPS will be written before any other frames? Or..? – some.birdie Feb 18 '13 at 11:10

1 Answers1

3

The mdat atom is a big blob of data that makes no sense on its own. To understand its contents, one must first parse the moov atom. You said that you decomposed the MOV files into atoms-- did you dig deeper than just moov, mdat, ftyp, and a few other top-level atoms? You have to parse the moov atom (lots of atoms in there) to get the data you need (extra codec data attached to the H.264 stsd atom which is combined with data from the H.264 chunks, which are indexed by the stco or co64 atoms).

Or use a library that already does all the tedious work for you.

Multimedia Mike
  • 12,660
  • 5
  • 46
  • 62
  • 3
    Hi Mike, I can't use other libraries because all libraries assumes that the file is sealed, and I'm parsing a file that is written while i read it. If I use only a single H264 video track, it's not hard to get the NAL unit from the 'mdat' - it's the only thing in there (Actually their header is replaced by their size). How can I use the 'moov' atom info in order to parse a file with also audio in it? Any good reference? Thanks! – Avishay Cohen Aug 08 '12 at 13:19