3

I'm trying to create an mp4 video using a camera that provides h264 raw compressed data (annex b format), without decoding or encoding operations.I use v4l2 to get frames from the camera and for each frame I fill an AVPacket and write it to the file using av_write_frame. It works well, but the video is not so fluid and using ffprobe I noticed that there are no B-frames in the video, but only I-frames and P-frames.

for(int i = 0; i<100; i++){

    // Dequeue the buffer
    if(ioctl(fd, VIDIOC_DQBUF, &bufferinfo) < 0)
        perror("Could not dequeue the buffer, VIDIOC_DQBUF");

    AVPacket packet;
    av_init_packet(&packet);

    packet.stream_index = videoStream->index;
    //buffer is allocated with mmap
    packet.data = (uint8_t*)buffer;
    //byteused is about 20/40 kB
    packet.size = bufferinfo.bytesused;

    packet.pts = (1.0 / 30) * 24000 * i;

    err = av_write_frame(outputFormatCtx, &packet);
    if(err < 0)
        cout << "Error write frame "<<endl;
    printf("Write frame  (size= %2d)\n", packet.size);

    if(ioctl(fd, VIDIOC_QBUF, &bufferinfo) < 0){
        perror("Could not queue buffer, VIDIOC_QBUF");
        return 1;
    }
    av_packet_unref(&packet);
}
err = av_write_trailer(outputFormatCtx);
if( err < 0)
    cout<<"Error : av_write_trailer()" << endl;

I don't understand if the problem is the capture or the file writing. Thanks.

Cristiano
  • 73
  • 1
  • 8
  • MP4 doesn’t use annex b. You must set the extradata in the codec context, and replace start codes with nalu sizes – szatmary Mar 15 '18 at 00:54
  • @szatmary I'm not using the codec context. Do I have encode this data with avcodec_send_frame/avcodec_receive_packet after setting properly the codec context? – Cristiano Mar 15 '18 at 10:32
  • No, you don’t need to reencode. You just need to context to store the extra data. – szatmary Mar 15 '18 at 13:26
  • Still doesn't work. This is my sps + pps: 0x00, 0x00 ,0x00,0x01, 0x67, 0x4d, 0x40, 0x1f, 0x96, 0x54, 0x03, 0xc0, 0x11, 0x3f, 0x2a, 0x00, 0x00, 0x00, 0x01, 0x68, 0xee, 0x38, 0x80 and I created the relative extradata: 0x01,0x4d, 0x40,0x1f,0xff,0xe1,0x00,0xb0,0x67, 0x4d, 0x40, 0x1f, 0x96, 0x54, 0x03, 0xc0, 0x11, 0x3f, 0x2a,0x01,0x00,0x04,0x68, 0xee, 0x38, 0x80. I also replaced the IDR start code with his size ( on 4 bytes). Nevertheless ffprobe output1.mp4 -show_frames | more' reports: No start code is found. Error splitting the input into NAL units. @szatmary – Cristiano Mar 16 '18 at 10:34
  • Post the resulting mp4. – szatmary Mar 16 '18 at 17:24
  • Ok now I can create the video, there was an error in the extradata. Do you want the ffprobe report? – Cristiano Mar 16 '18 at 17:35
  • Why would I want that? – szatmary Mar 16 '18 at 17:36
  • https://streamable.com/ajxlr – Cristiano Mar 16 '18 at 17:43
  • The container looks pretty good. I haven’t had time to do a deeper analysis, but it appears the issue is in the raw stream. – szatmary Mar 16 '18 at 18:46
  • I don't understand the lack of B frames.. maybe I'm badly parsing the stream or losing some NAL. At the moment every AVPacket is composed by only one NAL, because I have not found others start codes – Cristiano Mar 16 '18 at 18:56
  • B frame are optional. Why do you assume the stream would have them? – szatmary Mar 16 '18 at 18:57
  • Reading from camera with ffmpeg the video has also B frames, but probably ffmpeg does some decoding and encoding operations that I don't. – Cristiano Mar 16 '18 at 19:01
  • Ok, I looked again. Your mp4 as B frames also. You should set a dts on those. – szatmary Mar 16 '18 at 19:05
  • How? I'm using " ffprobe output1.mp4 -show_frames | more " to see every pict_type and they are only I and P. – Cristiano Mar 16 '18 at 19:09
  • The file you posted has b frames. Maybe streamable reencoded it. I’m sorry but I don’t have any more time to spend on this. When you find out more, open a new question and I can take a look. – szatmary Mar 16 '18 at 19:16
  • Thank you for everything. I've opened another question, I'd like to have your opinion when you can. https://stackoverflow.com/questions/49339046/adjust-pts-and-dts-before-mp4-video-creation – Cristiano Mar 17 '18 at 16:13
  • Hi @Cristiano Have you managed to solve your problem? Could you post complete code like a minimal working example? I'm recording the stream from a camera when the user pushes a button and I'd like to create the mp4 container on-the-fly as encoded frames arrive from my camera in raw h264 format. In this way I would avoid the post-processing step of an h264 file (which takes quite long right now as it runs on a rather slow embedded device). – remus Jul 12 '18 at 07:56
  • 1
    Hi @remus, this is my minimal working code https://pastebin.com/yG37YhWW. I have not yet set properly all the parameters , but for now it works well. – Cristiano Jul 12 '18 at 08:47
  • Many thanks! I made some changes to it because my camera has hardware h264 encoder and I'm getting the encoded frame in a callback. I can playback the file now in any player and the container seems fine. However, the duration is wrong. I'm recording for 10 seconds and it plays for 20 seconds (looks like slow motion). But the framerate is correct. I'll look into it. Thanks again! – remus Jul 12 '18 at 10:58
  • Try with `ffprobe filename.mp4 -show_frames | more` You can see the duration of every packets and their pts and dts. In my case the duration is 0.03333 (30 fps) while pts_time and dts_time increase by 0.03333 at a time. However I think their value could be the only problem of your wrong duration. – Cristiano Jul 12 '18 at 11:49
  • I found and fixed my bug: the callback function was called multiple times for a single frame, once with a low payload (size 6, 8 or 13 bytes - frame header?) and then with the actual encoded frame. I'm buffering the frame header and then create the AVPacket when I receive the actual encoded frame. And it runs fine. I'm getting the right FPS and bitrate when playing back. Many thanks for the working example - it helped. – remus Jul 16 '18 at 08:43
  • Hi again. Have you experienced that the mp4 file cannot be played by Windows Media Player in Windows 7? It plays well in any other player and also WMP in Windows 10 but it fails in WMP Windows 7. – remus Sep 28 '18 at 09:58
  • Hi, no I'm sorry, I'm only using vlc on linux. However I had the same problem with Android and I solved changing the compatibility option, check here the relative section trac.ffmpeg.org/wiki/Encode/H.264, maybe this could fix your problem too. – Cristiano Sep 29 '18 at 10:47

0 Answers0