4

I use ffmpeg's MPEG4 decoder. The decoder has CODEC_CAP_DELAY capability among others. It means the decoder will give me decoded frames with latency of 1 frame.

I have a set of MPEG4 (I- & P- )frames from AVI file and feed ffmpeg decoder with these frames. For the very first I-frame decoder gives me nothing, but decodes the frames successfully. I can force the decoder to get the decoded frame with the second call of avcodec_decode_video2 and providing nulls (flush it), but if I do so for each frame I get artifacts for the first group of pictures (e.g. second decoded P-frame is of gray color).

If I do not force ffmpeg decoder to give me decoded frame right now, then it works flawlessly and without artifacts.

Question: But is it possible to get decoded frame without giving the decoder next frame and without artifacts?

Small example of how decoding is implemented for each frame: 

        // decode
        int got_frame = 0;
        int err = 0;
        int tries = 5;
        do
        {
            err = avcodec_decode_video2(m_CodecContext, m_Frame, &got_frame, &m_Packet);
            /* some codecs, such as MPEG, transmit the I and P frame with a
            latency of one frame. You must do the following to have a
            chance to get the last frame of the video */
            m_Packet.data = NULL;
            m_Packet.size = 0;
            --tries;
        }
        while (err >= 0 && got_frame == 0 && tries > 0);

But as I said that gave me artifacts for the first gop.

Sergey
  • 357
  • 5
  • 16

2 Answers2

6

Use the "-flags +low_delay" option (or in code, set AVCodecContext.flags |= CODEC_FLAG_LOW_DELAY).

Ronald S. Bultje
  • 10,828
  • 26
  • 47
  • @ronald-s-bultje I have had so much problems dealing with delays, didn't even knew this flag existed. Looking at the docs all it says about it is Force low delay which doesn't tell me much. Looking at the source I only find it considered for h264 and some mpegX codes. Will it only work for those? Will it always discard all delay? or where can I get more information about what this flag does. Thanks in advance, this will save me some headaches. – Maxito Apr 17 '15 at 19:25
  • 1
    @maxito: I believe it will automatically re-enable delay when it encounters B-frames in MPEG-4. I haven't checked the other codecs, but I vaguely recall that for H264 (related to the more complicated pyramid frame-ordering structure), delay is interpreted as "max-delay" which means that if we encounter more queued frames than indicated delay, we assume we missed a frame (decoding error? packet drop?) and start outputting frames. I would recommend to read the source code to see what the flag does exactly. It's only supported in MPEG-1/2/4, H264 and VC1. – Ronald S. Bultje Apr 17 '15 at 19:50
  • Thank you for the information, will take a deeper look at the source. – Maxito Apr 17 '15 at 20:21
  • @ronald-s-bultje tried to use this flag for mpeg4 encoder - avcodec_encode_video2 started failing. But with decoder the flag works. – Sergey Apr 20 '15 at 10:04
  • For encoding, you'll also need to set some other flags, for example AVCodecContext.max_b_frames needs to be set to 0. Did you get any log messages, or did the encode just return an error value without any message? – Ronald S. Bultje Apr 20 '15 at 10:57
  • For me, it's working as expected for decoding. The only downside I found is that when setting the flag, _avcodec_decode_video2_ is taking way longer than before, which is not good for my purposes. – Maxito Apr 20 '15 at 23:19
  • Important note - if you set AVCodecContext.flags |= CODEC_FLAG_LOW_DELAY you will force the decoder to be single threaded! It took me a long time to find this. The decoder will ignore the -thread X and will only use a single thread for decoding which significantly impacts performance! – John Allard Apr 13 '18 at 05:25
  • Slice-threading (-thread_type slice) is compatible with low-delay decoding. – Ronald S. Bultje Nov 30 '20 at 14:52
0

I tested several options and "-flags low_delay" and "-probesize 32" is more important than others. bellow code worked for me.

AVDictionary* avDic = nullptr;
av_dict_set(&avDic, "flags", "low_delay", 0);
av_dict_set(&avDic, "probesize", "32", 0);

const int errorCode = avformat_open_input(&pFormatCtx, mUrl.c_str(), nullptr, &avDic);
keyvan moazami
  • 39
  • 5