1

I'm writing a screen capture application that uses the UWP Screen Capture API.

As a result, I get a callback each frame with a ID3D11Texture2D with the target screen or application's image, which I want to use as input to MediaFoundation's SinkWriter to create an h264 in mp4 container file.

There are two issues I am facing:

  • The texture is not CPU readable and attempts to map it fail
  • The texture has padding (image stride > pixel width * pixel format size), I assume

To solve them, my approach has been to:

  • Use ID3D11DeviceContext::CopyResource to copy the original texture into a new one created with D3D11_USAGE_STAGING and D3D11_CPU_ACCESS_READ flags
  • Since that texture too has padding, create a IMFMediaBuffer buffer wrapping it with MFCreateDXGISurfaceBuffer cast it to IMF2DBuffer and use IMF2DBuffer::ContiguousCopyTo to another IMFMediaBuffer that I create with MFCreateMemoryBuffer

So I'm basically copying around each and every frame two times, once on the GPU and once on the CPU, which works, but seems way inefficient.

What is a better way of doing this? Is MediaFoundation smart enough to deal with input frames that have padding?

Alberto
  • 403
  • 3
  • 11
  • You can check this question and the answers (c# but nonetheless a good explanation) for details on how to implement hardware h264 encoding that will directly use your texture as input (as Roman suggested): https://stackoverflow.com/questions/44402898/mf-sinkwriter-write-sample-failed – VuVirt Jul 17 '19 at 13:13

1 Answers1

0

The inefficiency comes from your attempt to Map rather than use the texture as video encoder input. MF H.264 encoder, which is in most cases a hardware encoder, can take video memory backed textures as direct input and this is what you want to do (setting up encoder respectively - see D3D/DXGI device managers).

Padding does not apply to texture frames. In case of padding in frames with traditional system memory data Media Foundation primitives are normally capable to process data with padding: Image Stride and MF_MT_DEFAULT_STRIDE, and other Video Format Attributes.

Roman R.
  • 68,205
  • 6
  • 94
  • 158