omxplayer AVPacket

Question

Hi Im trying decode video usin OpenMax component and hello_video example from GitHub. I simply edit video.c and replace "main frame while" on AVFormat -read frame packet for read not only *.h264 files . (I need read mp4, mov, mkv....).. So with test.h264 files and other files that I generate with ffmpeg, all works fine and on screen I can see movie.

ffmpeg -i file.mp4 -vcodec copy -vbsf h264_mp4toannexb out.h264

But if I open test.mp4 file I can't see pictures on the screen, and log in console show me that data read correctly and parsed to the input buffer of the video decoder correctly. Can any explain me why I can't see anything on the screen on second test?

    do{
    printf("###before DO!\n");
status=av_read_frame(pFormatCtx,&packet);
//only for video
if(packet.stream_index==*video_stream_index){
printf("=>Read frame, status: %d, index: %d, stream index: %d, packet duration: %d, size: %d\n",pstatus,index++,packet.stream_index,packet.duration,packet.size);
int psize=packet.size;
int preaded=0;
double pts=packet.duration;
while(psize!=0){
     buf = ilclient_get_input_buffer(video_decode, 130, 1);
      buf->nFlags = 0;
      buf->nOffset = 0;
      uint64_t val = (uint64_t)(pts == DVD_NOPTS_VALUE) ? 0 : pts;
     if(first_frame==true){buf->nFlags = OMX_BUFFERFLAG_STARTTIME;first_frame=false;}else{buf->nFlags = OMX_BUFFERFLAG_TIME_UNKNOWN;}
buf->nTimeStamp = ToOMXTime(val);
buf->nFilledLen = (psize > buf->nAllocLen) ? buf->nAllocLen : psize;
memcpy(buf->pBuffer, packet.data+preaded,buf->nFilledLen);
psize-=buf->nFilledLen;
preaded+=buf->nFilledLen;
     if(psize == 0){buf->nFlags|=OMX_BUFFERFLAG_ENDOFFRAME;printf("#######################################OMX_BUFFERFLAG_ENDOFFRAME\n");}
printf("=>BUFF size: %d\n",buf->nFilledLen);
OMX_ERRORTYPE r;
if(pstatus==0){if(r=OMX_EmptyThisBuffer(ILC_GET_HANDLE(video_decode), buf) != OMX_ErrorNone){status = -6;printf("Failed, OMX_EmptyThisBuffer, error: 0x%08x , buf allocate: %d, buf lenght: %d \n", r,buf->nAllocLen,buf->nFilledLen);break;}}
}//while psize
av_free_packet(&packet);
   }//if index  
}//do
while(pstatus==0);

Answer 1

log in console show me that data read correctly and parsed to the input buffer of the video decoder correctly

What do you mean by that? Most likely, your packets demuxed from mp4 are not in annex-b form, which is expected by OMX decoder, and which is the natural form of .h264 file (stream) format. So, after you read a NALU, replace the header with 00000001 (four bytes with one last bit on), and you may be lucky.

Other things to think about: decoder may or may not expect SPS/PPS with the first I-frame, and so on...

Update: LLVM test-suite has a C source file that does exactly this. You may also be interested in the reference AVC/h264 implementation that does the opposite conversion - from Annex B to NALU.