Find better VP8 parameters for robustness in UDP streaming with libav/ffmpeg

Question

I'm facing some problems in my video chat application, which is using libav libraries. I am sending 1080p videos encoded in VP8 as WebM container via UDP and it works quite well. Most of the time, the decoder on either side recovers from packet losses due to the transmission.

However at some point in time it just freezes and never recovers again. This happens on both sides eventually. I was searching for VP8 codec parameters to set for increased robustness, when sending over lossy transmission channels. And I combined some of which I found, in order to increase robustness. However, it still freezes after some time of video chat.

Here are the parameters I am currently using.

  pVidCodecCtx->codec_id     = AV_CODEC_ID_VP8;
  pVidCodecCtx->codec_type   = AVMEDIA_TYPE_VIDEO;
  pVidCodecCtx->width        = frmQ->pCodecCtx->width; //1920
  pVidCodecCtx->height       = frmQ->pCodecCtx->height; //1080
  pVidCodecCtx->time_base    = frmQ->pCodecCtx->time_base;
  pVidCodecCtx->pix_fmt      = PIX_FMT_YUV420P;
  pVidCodecCtx->qmin         = 4;
  pVidCodecCtx->qmax         = 56;
  pVidCodecCtx->bit_rate     = pVidCodecCtx->width * pVidCodecCtx->height * 6;
  pVidCodecCtx->slices       = 8;
  pVidCodecCtx->profile      = 3;
  pVidCodecCtx->thread_count = 3;
  pVidCodecCtx->keyint_min   = 5;
  av_dict_set(&pDictCodecOpts, "rc_lookahead", "0", 0);
  av_dict_set(&pDictCodecOpts, "quality", "realtime", 0);
  av_dict_set(&pDictCodecOpts, "deadline", "realtime", 0);
  av_dict_set(&pDictCodecOpts, "max-intra-rate", "0", 0);
  av_dict_set(&pDictCodecOpts, "qcomp", "0", 0);
  av_dict_set(&pDictCodecOpts, "default", "er", 0);
  av_dict_set(&pDictCodecOpts, "error_resilient", "er", 0);
  av_dict_set(&pDictCodecOpts, "partitions", "er", 0);

Most of the parameters I extracted from the ffmpeg code for the vpx encoder.

Do I also have to set parameters for the decoder in order to increase error resilience? Or am I missing some parameters in the encoder or setting them incorrectly. Any help or hints are greatly appreciated.

score 3 · Accepted Answer · answered Oct 24 '14 at 09:17

I'll answer my own question, because I eventually managed to get the video chat running without freezing.

Turns out, that these parameters are actually good for error resilience.

The problem was that some of the function calls e.g.

av_read_frame()
avformat_open_input()

in the decoder thread were blocking when the package was corrupted, which led to freezing of the video.

So what I ended up doing was to write a timer class and let the timer measure the time of execution of said functions. I wrote a interrupt callback function and passed it over to the format context of my decoder like so:

static int interrupt_cb (void *p)
{
    unsigned int expTime = 1000;
    Uint32 elapsedTime = pVidConfTimer.elapsedTimeInMs();
    if (elapsedTime > expTime) 
    {
       return 1;
    }
    return 0;
}

static const AVIOInterruptCB cb = {interrupt_cb, &dummy};
frmQ.pFormatCtx->interrupt_callback = cb;

This will return from a function if the time for execution takes longer than expTime. You can also pass a custom parameter via void *p to the callback function

In my decoding/display thread I just call something like

timer.tic();
ret = av_read_frame(...);
timer.reset();

if (ret<0)
{
    //received corrupted frame
    //reinitialize format context
    //open input
    //find decoder and open codec
    ...
}

Hope this helps anybody having similar problems.

Find better VP8 parameters for robustness in UDP streaming with libav/ffmpeg

1 Answers1