I'm having issues with WinRT ScreenCapture blocking/freezing my UI.
My app is looping through selected windows and takes a screenshot of each every few seconds. Every time a screenshot is taken the UI animation is lagging and sometimes the UI just freezes completely until I focus on another app which somehow releases the UI block.
Using this ScreenCapture repo as a reference - https://github.com/microsoft/Windows.UI.Composition-Win32-Samples/tree/master/dotnet/WPF/ScreenCapture
In order to add screenshot capability, I've adjusted the BasicCapture.cs accordingly -
public class BasicCapture : IDisposable
{
...
private Texture2D cpuTexture; // added this
private bool screenshotReady; // added this
public BasicCapture(IDirect3DDevice d, GraphicsCaptureItem i)
{
...
cpuTexture = CreateTexture2D(item.Size.Width, item.Size.Height); // added this
}
public void Dispose()
{
session?.Dispose();
framePool?.Dispose();
swapChain?.Dispose();
d3dDevice?.Dispose();
cpuTexture?.Dispose(); // added this
}
private Texture2D CreateTexture2D(int width, int height)
{
// create add texture2D 2D accessible by the CPU
var desc = new Texture2DDescription()
{
Width = width,
Height = height,
CpuAccessFlags = CpuAccessFlags.Read,
Usage = ResourceUsage.Staging,
Format = Format.B8G8R8A8_UNorm,
ArraySize = 1,
MipLevels = 1,
SampleDescription = new SampleDescription(1, 0),
};
return new Texture2D(d3dDevice, desc);
}
private void OnFrameArrived(Direct3D11CaptureFramePool sender, object args)
{
using (var frame = sender.TryGetNextFrame())
{
...
using (var backBuffer = swapChain.GetBackBuffer<SharpDX.Direct3D11.Texture2D>(0))
using (var bitmap = Direct3D11Helper.CreateSharpDXTexture2D(frame.Surface))
{
d3dDevice.ImmediateContext.CopyResource(bitmap, backBuffer);
if (screenshotReady == false)
{
// added this to copy the DirectX resource into the CPU-readable texture2D
d3dDevice.ImmediateContext.CopyResource(bitmap, cpuTexture);
screenshotReady = true;
}
}
}
}
public async Task<Image> ScreenshotWindow()
{
screenshotReady = false;
StartCapture();
//await for screenshot to be captured
while (screenshotReady == false) await Task.Delay(50);
// get IDirect3DSurface from texture
using var surf = Direct3D11Helper.CreateDirect3DSurfaceFromSharpDXTexture(cpuTexture);
// build a WinRT's SoftwareBitmap from this surface/texture
using var softwareBitmap = await SoftwareBitmap.CreateCopyFromSurfaceAsync(surf);
using var InMemoryStream = new InMemoryRandomAccessStream();
BitmapEncoder encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.JpegEncoderId, InMemoryStream);
encoder.SetSoftwareBitmap(softwareBitmap);
await encoder.FlushAsync();
using var stream = InMemoryStream.AsStream();
return Image.Load(stream);
}
}
And to the BasicSampleApplication.cs Iv'e added this -
public async Task<Image> ScreenshotWindow(GraphicsCaptureItem item)
{
capture = new BasicCapture(device, item);
var surface = capture.CreateSurface(compositor);
brush.Surface = surface;
var img = await capture.ScreenshotWindow();
StopCapture();
return img;
}
As I don't actually need to visually show the captured screen on my app, I instantiate the
BasicSampleApplication without any visual properties, and I do it once on the app startup -
// Create the compositor.
compositor = new Compositor();
_screenshotService = new BasicSampleApplication(compositor);
Now all that's left is to iterate through the selectedWindows and screenshot them -
private async Task ScreenshotSelectedWindows()
{
while (ScreenshotsEnabled)
{
foreach (var wnd in _selectedWindows)
{
GraphicsCaptureItem item = CaptureHelper.CreateItemForWindow(wnd.WindowHandler);
using var img = await _screenshotService.ScreenshotWindow(item);
}
await Task.Delay(TimeSpan.FromSeconds(10));
}
}
And now when I use ScreenshotSelectedWindows() it works, but as I initially said its making the UI animations lag, and sometimes the UI just freezes completely till I remove focus.
Iv'e tried calling this method from a background task -
Task.Run(()=>ScreenshotSelectedWindows());
For some reason doing that won't work, the OnFrameArrived callback is never triggered.
Iv'e also tried to instantiate the BasicSampleApplication on the background thread but creating the compositor on a background thread throws an error System.UnauthorizedAccessException: 'Access is denied. '.
Any tips on how can I achieve that screenshot goal without blocking/freezing my UI?
Thanks!
Edit - I'll add a link to a GitHub issue as it seems to be referencing my issue specifically, which I fail to translate to my code :D
https://github.com/microsoft/Windows.UI.Composition-Win32-Samples/issues/69#issuecomment-1498188216
