1

I am having to build a video topology manually, which includes using loading and configuring the mpeg2videoextension (decoder). Otherwise the default topoloader fails to resolve the video stream automatically. I am using the default topology loader to resolve the rest of the topology.

Since I am loading the decoder manually, the docs say that I am responsible to get the decoder the hardware acceleration manager. (This decoder is D3D11 Aware). If I create a DXGI device, then create manager in code, I can pass the manager to the decoder, and it seems to work.

The docs also say, however that "In a Media Session scenario, the video renderer creates the Direct3D 11 device."

If this is the case, how do I get a handle to that device? I assume I should be using that device in the device manager to pass into the decoder.

I'm going around in circles. All of the sample code uses IDirect3DDeviceManager9. I am unable to get those samples to work. So I decided to use 11. But I can't find any sample code that uses 11.

Can someone point me in the right direction?

Roman R.
  • 68,205
  • 6
  • 94
  • 158
Jerry Davis
  • 151
  • 1
  • 11

2 Answers2

1

Microsoft does not give a good solution for this challenge. Indeed, standard video renderer for Media Foundation is EVR and it is "aware" of Direct3D 9 only. So you cannot combine it with the decoder using common DXGI device manager. Newer Microsoft applications use a different Direct3D 11 aware renderer, which is not published as an API: you can take advantage of these rendering services as a part of wrapping APIs such as UWP or HTML5 media element playing video. MPEG-2 decoder extension targets primarily these scanarios leaving you with a problem if you are plugging this into older Media Foundation topologies.

I can think of a few solutions to this problems, none of which sound exactly perfect:

  1. Stop using EVR and use DX11VideoRenderer instead: Microsoft gives a starting point with this sample and you are own your own to establish required wiring to share DXGI device manager.

  2. Use multiple Direct3D devices and transfer video frames between the two; there should be graphics API interop to help transfer in efficient way, but overall this looks a sort of stupid work as of 2020 even though doable. This path looks more or less acceptable if you can accept performance hit from transfer through system memory, which makes things a tad easier to implement.

  3. Stop using MPEG-2 decoder extension and implement your own decoder on top of lower level DXVA2 API and implement hardware assisted decoder without fallback to software, in which case you have more control over using GPU services and fitting to renderer's device.

Roman R.
  • 68,205
  • 6
  • 94
  • 158
  • Thank you for the quick response. I acknowledge that this is probably the answer. However, I am going to wait a bit for a more cheerful assessment, before I mark as such. :) – Jerry Davis Dec 31 '20 at 14:20
1

I also second the recommendation to use DX11VideoRenderer as a means to get familiar with the D3D11-aware devices and managers. The DX11VideoRenderer code makes intensive use of ID3D11Device, ID3D11VideoDevice and IMFDXGIDeviceManager interfaces, and the absence of published API would not prevent you from learning how to build the D3D11-aware topology.

There is a minor inconvenience of blurred recommendations on testing the DX11VideoRenderer with the topoedit application that the project README file gives. I updated the topoedit code so that you can instantiate DX11VideoRenderer sink node and build topologies with this node. I believe this project would help you move in the right direction.

V.V.T
  • 231
  • 1
  • 7