1

I'm using the FFmpege.AutoGen library in my project and found a crash on a video using hardware decoding. So I went back to the example and tested it there and found the same result.

The example reports the following

Current directory: D:\...\Ruslan-B\FFmpeg.AutoGen\FFmpeg.AutoGen.Example\bin\Debug\net6.0
Running in 64-bit mode.
FFmpeg binaries found in: D:\...\Ruslan-B\FFmpeg.AutoGen\FFmpeg\bin\x64
FFmpeg version info: 5.0-full_build-www.gyan.dev
Use hardware acceleration for decoding?[n]
y
Select hardware decoder:
1. AV_HWDEVICE_TYPE_CUDA
2. AV_HWDEVICE_TYPE_DXVA2
3. AV_HWDEVICE_TYPE_QSV
4. AV_HWDEVICE_TYPE_D3D11VA
5. AV_HWDEVICE_TYPE_OPENCL
6. AV_HWDEVICE_TYPE_VULKAN
Selected [2]

Decoding...
[mpegts @ 0000012917fced00] parser not found for codec klv, packets or times may be invalid.
[h264 @ 000001293b424080] Reinit context to 1920x1088, pix_fmt: yuv420p
[mpegts @ 0000012917fced00] parser not found for codec klv, packets or times may be invalid.
[AVHWDeviceContext @ 000001293b9a6940] Using D3D9Ex device.
codec name: h264
[h264 @ 000001293b8a5040] Decoder GUIDs reported as supported:
[h264 @ 000001293b8a5040] {ee27417f-5e28-4e65-beea-1d26b508adc9}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {bf22ad00-03ea-4690-8077-473346209b7e}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {e07ec519-e651-4cd6-ac84-1370cceec851}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {bcc5db6d-a2b6-4af0-ace4-adb1f787bc89}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {1b81bea4-a0c7-11d3-b984-00c04f2e73c5}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {1b81be68-a0c7-11d3-b984-00c04f2e73c5}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {d79be8da-0cf1-4c81-b82a-69a4e236f43d}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {f9aaccbb-c2b6-4cfc-8779-5707b1760552}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {705b9d82-76cf-49d6-b7e6-ac8872db013c}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {c528916c-c0af-4645-8cb2-372b6d4adc2a}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {07cfaffb-5a2e-4b99-b62a-e4ca53b6d5aa}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {91cd2d6e-897b-4fa1-b0d7-51dc88010e0a}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {90b899ea-3a62-4705-88b3-8df04b2744e7}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {442b942a-b4d9-4940-bc45-a882e5f919f3}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {8c56eb1e-2b47-466f-8d33-7dbcd63f3df2}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {5b11d51b-2f4c-4452-bcc3-09f2a1160cc0}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {75fc75f7-c589-4a07-a25b-72e03b0383b3}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]  1[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {107af0e0-ef1a-4d19-aba8-67a163073d13}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]  1[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {463707f8-a1d0-4585-876d-83aa6d60b89e}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {a4c749ef-6ecf-48aa-8448-50a7a1165ff7}[h264 @ 000001293b8a5040]  1[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {76988a52-df13-419a-8e64-ffcf4a336cf5}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {80a3a7bd-89d8-4497-a2b8-2126af7e6eb8}[h264 @ 000001293b8a5040]  1[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {a74ccae2-f466-45ae-86f5-ab8be8af8483}[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {1b81be94-a0c7-11d3-b984-00c04f2e73c5}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {1b81bea2-a0c7-11d3-b984-00c04f2e73c5}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] {49761bec-4b63-4349-a5ff-87ffdf088466}[h264 @ 000001293b8a5040]  0[h264 @ 000001293b8a5040]
[h264 @ 000001293b8a5040] No decoder device for codec found
[h264 @ 000001293b8a5040] Failed setup for format dxva2_vld: hwaccel initialisation returned error.
[h264 @ 000001293b8a5040] Reinit context to 1920x1088, pix_fmt: yuv420p
Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
   at FFmpeg.AutoGen.ffmpeg+<>c.<.cctor>b__7_466(FFmpeg.AutoGen.AVFrame*, FFmpeg.AutoGen.AVFrame*, Int32)
   at FFmpeg.AutoGen.ffmpeg.av_hwframe_transfer_data(FFmpeg.AutoGen.AVFrame*, FFmpeg.AutoGen.AVFrame*, Int32)
   at FFmpeg.AutoGen.Example.VideoStreamDecoder.TryDecodeNextFrame(FFmpeg.AutoGen.AVFrame ByRef)
   at FFmpeg.AutoGen.Example.Program.DecodeAllFramesToImages(FFmpeg.AutoGen.AVHWDeviceType)
   at FFmpeg.AutoGen.Example.Program.Main(System.String[])

The errors/messages

...
[h264 @ 0000013533bfb040] No decoder device for codec found
[h264 @ 0000013533bfb040] Failed setup for format dxva2_vld: hwaccel initialisation returned error.
[h264 @ 0000013533bfb040] Reinit context to 1920x1088, pix_fmt: yuv420p

are reported in the call to

error = ffmpeg.avcodec_receive_frame(_pCodecContext, _pFrame);

The result (error) of ffmpeg.avcodec_receive_frame is 0. so there's no obvious error. But the messages make it obviously that the hardware decoder can't handle the video.

The exception is thrown in the call to ffmpeg.av_hwframe_transfer_data()

if (_pCodecContext->hw_device_ctx != null)
{
    ffmpeg.av_hwframe_transfer_data(_receivedFrame, _pFrame, 0).ThrowExceptionIfError();
    frame = *_receivedFrame;
}

My question is how can I determine this beforehand and not use the hardware decoder? i.e. fall back to the software decoder.

Edit: I've found that _pCodecContext->hwaccel is null after the call to ffmpeg.avcodec_receive_frame(_pCodecContext, _pFrame); when for the Truck.ts video but for videos where the hardware decoder is working _pCodecContext->hwaccel is non-null.

Updating the code to

if (_pCodecContext->hw_device_ctx != null && _pCodecContext->hwaccel != null)
{
    ffmpeg.av_hwframe_transfer_data(_receivedFrame, _pFrame, 0).ThrowExceptionIfError();
    frame = *_receivedFrame;
}

Seems to be enough to catch the error. Is the right way to check that the hardware decoder is working or is there a better approach?

Bernie
  • 15
  • 5
  • If this is actually a programming question, you should tag the question with the language you're working with, and also remove emphasis on the end-user output which makes it seem like not a programming question. – miken32 Aug 03 '22 at 20:23
  • Yes it's a programming question. hence the link to the [source code](https://github.com/Ruslan-B/FFmpeg.AutoGen/tree/master/FFmpeg.AutoGen.Example) and the included sections of code where the problem is raised when viewing the linked video. I've added tags for C and C# since the C# code is an autogenerated wrapper around the C API. But really I think a solution in most languages would point to a how to do similar in C/C#. – Bernie Aug 04 '22 at 06:57
  • This is a good question. I was trying similarly to find a proper way to handle it once and not checking it on every frame. However it does not seem possible. By the way, one tip, some videos might require to allow a profile mismatch to be able to hardware decode them. However, this can cause issues for other videos. Eg;- codecCtx->hwaccel_flags |= AV_HWACCEL_FLAG_ALLOW_PROFILE_MISMATCH; – SuRGeoNix Sep 01 '22 at 09:03
  • Thanks for the tip @surgeonix. The [FFmpeg docs](https://ffmpeg.org/doxygen/3.4/group__lavc__hwaccel.html#ga7403e00bf9dc00ed3d06579611075994) kind of scare me off. "**If the stream is actually not supported then the behaviour is undefined, and may include returning entirely incorrect output while indicating success.**" Stability is key for me. – Bernie Sep 06 '22 at 07:35

1 Answers1

1

You must check _pFrame->hw_frames_ctx are not null.
If HW failed, hw_frames_ctx in frame will be null

Trương Quốc Khánh
  • 534
  • 1
  • 7
  • 25
  • Thanks, I've updated the code to use `if (_pFrame->hw_frames_ctx != null)` and this is working the same. – Bernie Sep 06 '22 at 07:29