2

I am trying streaming with ffmpeg using the information I found here.

  • Server

    ffmpeg -f dshow -i video="john":audio="doe" -vcodec libx264
-preset ultrafast -tune zerolatency -acodec aac -f mpegts udp://localhost:1234

  • Client

    ffmpeg -i udp://localhost:1234?listen test.mp4

The client will actually live stream the video, but for testing purposes I am writing the output to a file for now.

Here comes the interesting part. I began the stream(video and audio) in a dark room, everything is fine. I turn on the room's lights and the client cries with many errors. I later checked the output, it is very corrupted indeed on the light turned-on parts. What could be the reason behind this ? I am posting the errors here if it could be of assistance.

Some of the errors:

udp://localhost:1234?listen: corrupt decoded frame in stream 0trate= 845.6kbits/s dup=1250 drop=0 speed=1.08x
    Last message repeated 1 times
[h264 @ 0000026c4af69400] Invalid NAL unit 1, skipping.7.61 bitrate= 864.7kbits/s dup=1441 drop=0 speed=1.07x
[h264 @ 0000026c4af69400] Invalid level prefix
[h264 @ 0000026c4af69400] error while decoding MB 18 8
[h264 @ 0000026c4af69400] concealing 911 DC, 911 AC, 911 MV errors in P frame
[h264 @ 0000026c4af4b780] Invalid NAL unit 1, skipping.
[h264 @ 0000026c4af4b780] negative number of zero coeffs at 10 14
[h264 @ 0000026c4af4b780] error while decoding MB 10 14
[h264 @ 0000026c4af4b780] concealing 679 DC, 679 AC, 679 MV errors in P frame
udp://localhost:1234?listen: corrupt decoded frame in stream 0
    Last message repeated 1 times
[h264 @ 0000026c4ba1e1c0] Invalid NAL unit 1, skipping.8.21 bitrate= 858.0kbits/s dup=1460 drop=0 speed=1.07x
[h264 @ 0000026c4ba1e1c0] out of range intra chroma pred mode
[h264 @ 0000026c4ba1e1c0] error while decoding MB 34 18
[h264 @ 0000026c4ba1e1c0] concealing 495 DC, 495 AC, 495 MV errors in P frame
[h264 @ 0000026c4bb16840] top block unavailable for requested intra mode
[h264 @ 0000026c4bb16840] error while decoding MB 10 0
[h264 @ 0000026c4bb16840] concealing 160 DC, 160 AC, 160 MV errors in P frame
udp://localhost:1234?listen: corrupt decoded frame in stream 0
[h264 @ 0000026c4bb16cc0] Invalid NAL unit 0, skipping.8.58 bitrate= 853.9kbits/s dup=1463 drop=0 speed=1.06x
[h264 @ 0000026c4bb16cc0] corrupted macroblock 16 28 (total_coeff=-1)
[h264 @ 0000026c4bb16cc0] error while decoding MB 16 28
[h264 @ 0000026c4bb16cc0] concealing 113 DC, 113 AC, 113 MV errors in P frame
[h264 @ 0000026c4bb17140] cbp too large (84) at 12 0
[h264 @ 0000026c4bb17140] error while decoding MB 12 0
[h264 @ 0000026c4bb17140] concealing 160 DC, 160 AC, 160 MV errors in P frame
[mpegts @ 0000026c4aeb8d80] PES packet size mismatch

Edit: As suggestion, I added these parameters to the server.

-b:v 1M -bufsize 2M

Now the errors are mostly gone. Except one time where it seemed like it occured just as I was turning off the lights, although I could not reproduce it after trying 4-5 times.

Errors:

[h264 @ 000002646f0d2f80] cbp too large (118) at 23 1422.25 bitrate= 659.5kbits/s dup=368 drop=0 speed=1.33x
[h264 @ 000002646f0d2f80] error while decoding MB 23 14
[mpegts @ 000002646e578d80] PES packet size mismatch
[h264 @ 000002646f0d2f80] concealing 666 DC, 666 AC, 666 MV errors in P frame
[h264 @ 000002646f0d0f80] concealing 160 DC, 160 AC, 160 MV errors in P frame
[aac @ 000002646e64d640] Number of bands (59) exceeds limit (43).
Error while decoding stream #0:1: Invalid data found when processing input
[aac @ 000002646e64d640] Multiple frames in a packet.
[aac @ 000002646e64d640] Reserved bit set.
[aac @ 000002646e64d640] Number of bands (31) exceeds limit (29).
Error while decoding stream #0:1: Invalid data found when processing input
[h264 @ 000002646e5e11c0] concealing 160 DC, 160 AC, 160 MV errors in P frame
udp://localhost:1234?listen: corrupt decoded frame in stream 0trate= 633.1kbits/s dup=368 drop=0 speed=1.35x
    Last message repeated 2 times

The errors decreased because I increased the bitrate ? (I don't know the default bitrate by the way). Bright decoding fails because it carries more information, data ? (same in raw video, but maybe file is heavier encoded bright frames)

Max Paython
  • 1,595
  • 2
  • 13
  • 25
  • If you start with the lights on, do you have this problem? Or is it only when transitioning from dark to light? And, by that same token, if you rapidly point the camera at a totally new scene, do you have the issue? – Brad Jul 28 '19 at 03:28
  • @Brad Problem exists when lights started on and goes away when turned off. – Max Paython Jul 28 '19 at 03:30
  • 1
    Ah, ok, so it's not an issue then with i-frames then. Sorry, can't be of more help... this one is out of the depth of what I know. – Brad Jul 28 '19 at 03:32
  • What about forcing a fixed bitrate? Is the problem reproducible above a certain bitrate? Might be an issue with the packetization, and with the lights on, it's taking more bits to encode your scene? – Brad Jul 28 '19 at 03:33
  • @Brad Please check the edit. – Max Paython Jul 28 '19 at 03:48
  • Can you fiddle with `pkt_size`? Try `udp://localhost:1234?pkt_size=1316` I really doubt fragmentation is going to be an issue on localhost though. Hmm... – Brad Jul 28 '19 at 03:56
  • @Brad Does this mean I can't use CRF value because it results in variable bitrate ? – Max Paython Jul 28 '19 at 03:58
  • Variable bitrate should be fine, I think what you've just found is that the problem is related to the bitrate itself, and it just so happens that your tests previously were using more bandwidth when you turned your lights on, which makes sense. – Brad Jul 28 '19 at 04:02
  • @Brad But only values I tested in the edit works relatively fine. Increasing or decreasing it did much worse jobs and gave errors (interesting). I also tried -crf 19, which also gave bad results. – Max Paython Jul 28 '19 at 04:13

1 Answers1

0

On your client side (I see you're using FFmpeg for this), add:

-flags2 showall

So your CLI will look like:

ffmpeg -flags2 showall -i udp://localhost:1234?listen test.mp4

That allows frames before the first keyframe to be rendered instead of being discarded.

Some drone cameras (DJI) produce footage with the same issue that can only be corrected by that flag.

Dennis Mungai
  • 1,579
  • 1
  • 15
  • 28