C# UWP WriteableBitmap to MediaClip conversion

Question

I'm working with UWP app. I'm generating set of WriteableBitmap's and after that, i want to add it to MediaComposition and play in video player. I found one way how to do this, but it's really slow. Here is my code:

var clip = await MediaClip.CreateFromImageFileAsync(await 
WriteableBitmapToStorageFile(wb), new TimeSpan(0, 0, 0, 0, 1));
(...)

private static async Task<StorageFile> WriteableBitmapToStorageFile(WriteableBitmap WB)
    {
        string FileName = "MyFile.";
        Guid BitmapEncoderGuid = BitmapEncoder.TiffEncoderId;
        FileName += "tiff";

        var file = await Windows.Storage.ApplicationData.Current.TemporaryFolder
            .CreateFileAsync(FileName, CreationCollisionOption.GenerateUniqueName);

        using (IRandomAccessStream stream = await file.OpenAsync(FileAccessMode.ReadWrite))
        {
            BitmapEncoder encoder = await BitmapEncoder.CreateAsync(BitmapEncoderGuid, stream);
            Stream pixelStream = WB.PixelBuffer.AsStream();
            byte[] pixels = new byte[pixelStream.Length];
            await pixelStream.ReadAsync(pixels, 0, pixels.Length);
            encoder.SetPixelData(BitmapPixelFormat.Bgra8, BitmapAlphaMode.Ignore,
                      (uint)WB.PixelWidth,
                      (uint)WB.PixelHeight,
                      96.0,
                      96.0,
                      pixels);
            await encoder.FlushAsync();
        }
        return file;
    }

Function "WriteableBitmapToStorageFile" takes almost 400ms to proceed WriteableBitmap. Is there any option to do it faster? I know, that MediaClip can be created from IDirect3DSurface but i don't know how to convert to it from WriteableBitmap.

Ok as @Johnny Westlake mentioned i used CanvasBitmap to draw my images. But now, after drawing many images only last image is displayed. Code:

public static CanvasDevice device = new CanvasDevice();
public static CanvasRenderTarget renderer = new CanvasRenderTarget(device, 2048, 1080, 96);

public static CanvasBitmap cb = CanvasBitmap.CreateFromBytes(device, new byte[64*64*4], 64, 64, Windows.Graphics.DirectX.DirectXPixelFormat.B8G8R8A8UIntNormalizedSrgb);

public static Rect dest = new Rect
    {
        Width = 2048,
        Height = 1080
    };

public static Rect src = new Rect
    {
        Width = 64,
        Height = 64
    };

private static CanvasBitmap GenerateHeatmap(byte[] renderedHeatmap )
{   
    cb.SetPixelBytes(renderedHeatmap);

    using (var ds = renderer.CreateDrawingSession())
        {
            ds.DrawImage(cb, dest, src, 1, CanvasImageInterpolation.Cubic, CanvasComposite.Copy);
    }
        return renderer;
    }

(...)

Later in the code:

cb = GenerateHeatmap(renderedHeatmap);
MediaOverlay mediaOverlay = new MediaOverlay(MediaClip.CreateFromSurface(cb, new TimeSpan(0, 0, 0, 0, 1)));
mediaOverlay.Position = overlayPosition;
mediaOverlay.Opacity = heatmapOpacity;

return mediaOverlay;

After i did it many times at the end there was only one image in MediaOverlays

Is there anyway you can use a Win2D CanvasBitmap instead of a writeablebitmap? They're editable, drawable in XAML, hardware accelerated, and are natively IDirect3DSurface's. If you can't, you can create a new CanvasBitmap from the WB's PixelBuffer using a CanvasBitmap.CreateFromBytes overload, but this is very wasteful compared to having a CanvasBitmap in the first place. — Johnny Westlake, Jun 27 '18 at 14:51
I will try your solution and let you know if it works. Thanks :) — Konrad Śledziewski, Jun 27 '18 at 15:00

score 0 · Answer 1 · answered Jun 29 '18 at 03:06

To create a video file by WriteableBitmap and MediaClip which created from IDirect3DSurface, you can try the following code using CanvasBitmap.

private async void CreateVideoFromWritableBitmapAsync(List<WriteableBitmap> WBs)
{
    var mediaComposition = new MediaComposition();
    foreach (WriteableBitmap WB in WBs)
    {
        var pixels = WB.PixelBuffer.ToArray();
        CanvasBitmap bitmap = CanvasBitmap.CreateFromBytes(CanvasDevice.GetSharedDevice(), pixels,
                64, 64, DirectXPixelFormat.B8G8R8A8UIntNormalized);
        MediaClip mediaClip = MediaClip.CreateFromSurface(bitmap, TimeSpan.FromSeconds(1));
        mediaComposition.Clips.Add(mediaClip);
    }
    //Save the video
    var file =await ApplicationData.Current.LocalFolder.CreateFileAsync("WBVideo.mp4");
    await mediaComposition.SaveAsync(file);
    mediaComposition=await MediaComposition.LoadAsync(file);
    await mediaComposition.RenderToFileAsync(file);
}

C# UWP WriteableBitmap to MediaClip conversion

1 Answers1