3

I am receiving the MPEG-TS (MPEG transport stream) packets with the multiplexed H.264 video and AAC audio streams. I need to be able to show the audio and video on the Android phone. My assumption is that I need:

  • MPEG-TS de-multiplexer
  • AAC decoder
  • H.264 decoder
  • Synchronize the audio and video playback

Assuming that I am right then (in Android 2.x) MPEG-TS de-multiplexer is not part of the OS and must be ported, both AAC and H.264 decoder are part of the Android OS, but I am not sure if they have interface, which allows passing the data in buffers and if they allow mutual timing synchronization. In the worst case those components must be ported here as well.

Can you give me some advices where to start? I was thinking about the FFMPEG porting. Are there any other ways?

Regards, STeN

STeN
  • 6,262
  • 22
  • 80
  • 125

4 Answers4

2

Android 4.x has OpenMAX which can play TS with H264 and AAC. You don't even need to worry about synchronisation of audio and video.

Look at the nativemedia sample in the NDK.

If you want to support previous versions of Android, then ffmpeg might be a good choice, but it the maximum it can give you is just decoded video frames in RGB or any other format and decoded audio in PCM. Then you will have to implement renderer and audio playback yourself. I would recommend reading this tutorial - http://dranger.com/ffmpeg/. It is not android specific but it will give you idea how video play works.

Dmitry
  • 2,837
  • 1
  • 30
  • 48
  • Hi, we already have finalized the FFMPEG based player for HLS (H.264/AAC) for Android and it works well! The effort we invest to have the really working solutions was about 3-4 man/month of really experienced developer. The problem is that the software decoding on ARM6/VFP or on ARM7 without NEON or TEGRA is not as fast as it can be with the HW support. So now we are looking for the StageFright integration on 2.3x - so far no luck - see e.g. there: http://stackoverflow.com/questions/11266891/how-to-enable-hw-decoder-via-stagefright-in-ffmpeg-android-port – STeN Aug 19 '12 at 12:48
  • 1
    Using standard APIs on 2.3 you can at least hardware-accelerate AAC decoding with OpenSL ES. – Dmitry Aug 20 '12 at 12:33
1

You may refer to the android-ffmpeg project on github.

https://github.com/guardianproject/android-ffmpeg

ciphor
  • 8,018
  • 11
  • 53
  • 70
1

In Gingerbread ( 2.3 ), actually there is a MPEG TS parser in the stagefright framework that you could use. Also, I believe it is well integrated with H264 and AAC decoders. MPEG TS parser is not advertised anywhere but the support is silently sitting there. I believe they have brought it to support Apple HTTP Live streaming in HC or later version but the code is sitting there in the Gingerbread ( 2.3 ) codebase as well. With a minor modification in the framework, you can playback http live streaming ( which actually sends TS packets). I guess the above information would be helpful for you.

Vibgyor

Mukesh
  • 242
  • 1
  • 7
  • Hi, do you think it can be back ported to 2.1 and 2.2 as well? I.e. back porting the MPEG TS parser from stagefright framework and connecting it 'somehow' with the H.264/AAC decoders? – STeN Feb 15 '12 at 06:00
  • Hi, MPEG TS parser code is integrated in the stagefright framework in the 2.3 release. It may not be that difficult to back port it in the 2.2 as 2.2 was also having support of stagefright framework but I guess it will be very difficult to back port it in 2.1 as stagefright framework is not there in 2.1 and you have opencore framework only there. – Mukesh Feb 15 '12 at 06:29
  • Vibgyor, you've alluded to this small fix to gingerbread to enable mpeg TS support. Do you have a link detailing this? – idlethread Apr 03 '12 at 11:24
0

(DISCLAIMER: I'm personally involved in developing the free and open source program linked below)

A static version of FFMpeg (both library and commandline) is provided by ZShaolin http://dyne.org/software/zshaolin also contains other media conversion tools.

Its use can facilitate scripting experiments without having to compile FFMpeg from scratch.

Jaromil
  • 76
  • 5
  • 2
    Welcome to Stack Overflow! Thanks for posting your answer! Please be sure to read the [FAQ on Self-Promotion](http://stackoverflow.com/faq#promotion) carefully. Also note that it is *required* that you post a disclaimer every time you link to your own site/product. – Andrew Barber Sep 10 '12 at 23:49
  • ACK. Thanks for the note. Still I believe this infois useful: AFAIK there aren't any other portable binaries of FFMpeg statically-compiled for Android and there is quite a demand for those on your forum.No self promotion intended here... I'm just trying to "live clean and let your work be seen". – Jaromil Sep 12 '12 at 13:19