1

I need to transfer frames of desktop image data over the network, that were captured with Desktop Duplication API, but there is one problem: size of raw frame data, in FullHD with DXGI_FORMAT_B8G8R8A8_UNORM format, approximatly equal to 8 MB, that too large size, especially for one frame.

Is there something good method to compress the frame, of desktop image, data from D3D11_MAPPED_SUBRESOURCE and get the size at least in KB? Maybe some kind of conversion from bmp to jpeg?

I've already tried LZ4 compression, with LZ4_compress_default method, and got byte array with 3.4 MB size, that is still too much for frame.

KKomrade
  • 95
  • 4
  • 1
    You need to use some form of lossy compression like ``JPG`` if something lossless like ``PNG`` doesn't meet your size requirements. Note you can use [Windows Imaging Component](https://learn.microsoft.com/en-us/windows/win32/wic/-wic-lh) to do this. For some example code, see [ScreenGrab](https://github.com/microsoft/DirectXTex/tree/master/ScreenGrab). – Chuck Walbourn Nov 18 '19 at 02:36

1 Answers1

1

...large size, especially for one frame

Since you are doing video and network transmission is in question, the only real choices for you are H.264 and H.265 video compression - those available implemented in hardware assisted encoders on video adapters. Microsoft Media Foundation API (Transforms) and video hardware vendor specific SDKs offer related functionality.

Windows Imaging Component (WIC) for individual images will be both slower and less efficient in compression ratio. Although it could be a good choice in the case individual snapshots are the only data to transmit, as opposed to video.

Also, an easy way to check out support for video encoding: http://alax.info/blog/1609

The tools enumerates the transforms and provides details, similar to Enumerating Media Foundation Transforms (MFTs) application

Roman R.
  • 68,205
  • 6
  • 94
  • 158
  • What method of compression can you advise, if screen capture implemented on clients and frames are sended to server over network within **VDI**? In this case **client can't see real hardware** and, i suppose, **using of Media Foundation**, with H.264 encoder, should be realized on **server side**, where we will use received frames to render the video. On the client side, perhaps, it might be better to use something to compress the raw data and send it over the network. – KKomrade Nov 18 '19 at 13:07
  • 1
    Not sure how VDI fits here. If the desktop image is composed on VD server in virtualized environment then it's perhaps server has to deal with video compression. And then - yes - it might have hard time accessing the GPU for compression services. – Roman R. Nov 18 '19 at 13:46