How to get byte array from SoftwareBitmap

Question

Hi I need help of how to get the byte array from a SoftwareBitmap in C# UWP, so that I can send it via TCP socket.

I also have access to a "VideoFrame previewFrame" object which is where I get the SoftwareBitmap from.

I have seen online to do something like the following, however UWP does not support the wb.SaveJpeg(...). Unless I am missing something?

MemoryStream ms = new MemoryStream();
WriteableBitmap wb = new WriteableBitmap(myimage);
wb.SaveJpeg(ms, myimage.PixelWidth, myimage.PixelHeight, 0, 100);
byte [] imageBytes = ms.ToArray();

Any help, or pointers in the right direction, would be greatly appreciated.

Thanks, Andy

Answer 1

Sure, you can access an encoded byte[] array from a SoftwareBitmap.

See this example, extracting a jpeg-encoded byte[]:

// includes BitmapEncoder, which defines some static encoder IDs
using Windows.Graphics.Imaging;


private async void PlayWithData(SoftwareBitmap softwareBitmap)
{
    // get encoded jpeg bytes
    var data = await EncodedBytes(softwareBitmap, BitmapEncoder.JpegEncoderId);

    // todo: save the bytes to a DB, etc
}

private async Task<byte[]> EncodedBytes(SoftwareBitmap soft, Guid encoderId)
{
    byte[] array = null;

    // First: Use an encoder to copy from SoftwareBitmap to an in-mem stream (FlushAsync)
    // Next:  Use ReadAsync on the in-mem stream to get byte[] array

    using (var ms = new InMemoryRandomAccessStream())
    {
        BitmapEncoder encoder = await BitmapEncoder.CreateAsync(encoderId, ms);
        encoder.SetSoftwareBitmap(soft);

        try
        {
            await encoder.FlushAsync();
        }
        catch ( Exception ex ){ return new byte[0]; }

        array = new byte[ms.Size];
        await ms.ReadAsync(array.AsBuffer(), (uint)ms.Size, InputStreamOptions.None);
    }
    return array;
}

Answer 2

to get byte array from SoftwareBitmap you can use "SoftwareBitmap.CopyToBuffer"

But, first you need:

using System.Runtime.InteropServices.WindowsRuntime;

because of method AsBuffer() to byte[]

...

StorageFile file = await StorageFile.GetFileFromPathAsync(ImageFilePath);
using (IRandomAccessStream fileStream = await File.OpenAsync(FileAccessMode.Read),
                                           memStream = new InMemoryRandomAccessStream())
  {
  // Open a Stream and decode a JPG image
  BitmapDecoder decoder = await BitmapDecoder.CreateAsync(fileStream);
  var softwareBitmap = await decoder.GetSoftwareBitmapAsync();

  byte [] imageBytes = new byte[4*decoder.PixelWidth*decoder.PixelHeight];
  softwareBitmap.CopyToBuffer(imageBytes.AsBuffer());
  //...  now you can use the imageBytes[]
}

Answer 3

as far I know you cant do it. but you can work with SoftwareBitmap. see examples: https://msdn.microsoft.com/en-us/library/windows/apps/mt244351.aspx (SoftwareBitmap is private field of SoftwareBitmapSource.. .just read it via reflection... maybe this is totally wrong suggestion)

private async void SaveSoftwareBitmapToFile(SoftwareBitmap softwareBitmap, StorageFile outputFile)
{
    using (IRandomAccessStream stream = await outputFile.OpenAsync(FileAccessMode.ReadWrite))
    {
        // Create an encoder with the desired format
        BitmapEncoder encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.JpegEncoderId, stream);

        // Set the software bitmap
        encoder.SetSoftwareBitmap(softwareBitmap);

        // Set additional encoding parameters, if needed
        encoder.BitmapTransform.ScaledWidth = 320;
        encoder.BitmapTransform.ScaledHeight = 240;
        encoder.BitmapTransform.Rotation = Windows.Graphics.Imaging.BitmapRotation.Clockwise90Degrees;
        encoder.BitmapTransform.InterpolationMode = BitmapInterpolationMode.Fant;
        encoder.IsThumbnailGenerated = true;

        try
        {
            await encoder.FlushAsync();
        }
        catch (Exception err)
        {
            switch (err.HResult)
            {
                case unchecked((int)0x88982F81): //WINCODEC_ERR_UNSUPPORTEDOPERATION
                                                 // If the encoder does not support writing a thumbnail, then try again
                                                 // but disable thumbnail generation.
                    encoder.IsThumbnailGenerated = false;
                    break;
                default:
                    throw;
            }
        }

        if (encoder.IsThumbnailGenerated == false)
        {
            await encoder.FlushAsync();
        }


    }
}

Answer 4

The Get Preview Frame UWP camera sample gets a camera frame as a SoftwareBitmap and manipulates the pixels (in-place) on it through an array, and can save it as a JPEG afterwards. If I understood your question correctly, all the code you need should be there.

Basically, this should be most of it (including some of the camera code):

private async Task GetPreviewFrameAsSoftwareBitmapAsync()
{
    // Get information about the preview
    var previewProperties = _mediaCapture.VideoDeviceController.GetMediaStreamProperties(MediaStreamType.VideoPreview) as VideoEncodingProperties;

    // Create the video frame to request a SoftwareBitmap preview frame
    var videoFrame = new VideoFrame(BitmapPixelFormat.Bgra8, (int)previewProperties.Width, (int)previewProperties.Height);

    // Capture the preview frame
    using (var currentFrame = await _mediaCapture.GetPreviewFrameAsync(videoFrame))
    {
        // Collect the resulting frame
        SoftwareBitmap previewFrame = currentFrame.SoftwareBitmap;

        // Add a simple green filter effect to the SoftwareBitmap
        EditPixels(previewFrame);
    }
}

private unsafe void EditPixels(SoftwareBitmap bitmap)
{
    // Effect is hard-coded to operate on BGRA8 format only
    if (bitmap.BitmapPixelFormat == BitmapPixelFormat.Bgra8)
    {
        // In BGRA8 format, each pixel is defined by 4 bytes
        const int BYTES_PER_PIXEL = 4;

        using (var buffer = bitmap.LockBuffer(BitmapBufferAccessMode.ReadWrite))
        using (var reference = buffer.CreateReference())
        {
            // Get a pointer to the pixel buffer
            byte* data;
            uint capacity;
            ((IMemoryBufferByteAccess)reference).GetBuffer(out data, out capacity);

            // Get information about the BitmapBuffer
            var desc = buffer.GetPlaneDescription(0);

            // Iterate over all pixels
            for (uint row = 0; row < desc.Height; row++)
            {
                for (uint col = 0; col < desc.Width; col++)
                {
                    // Index of the current pixel in the buffer (defined by the next 4 bytes, BGRA8)
                    var currPixel = desc.StartIndex + desc.Stride * row + BYTES_PER_PIXEL * col;

                    // Read the current pixel information into b,g,r channels (leave out alpha channel)
                    var b = data[currPixel + 0]; // Blue
                    var g = data[currPixel + 1]; // Green
                    var r = data[currPixel + 2]; // Red

                    // Boost the green channel, leave the other two untouched
                    data[currPixel + 0] = b;
                    data[currPixel + 1] = (byte)Math.Min(g + 80, 255);
                    data[currPixel + 2] = r;
                }
            }
        }
    }
}

And declare this outside your class to enable the GetBuffer method:

[ComImport]
[Guid("5b0d3235-4dba-4d44-865e-8f1d0e4fd04d")]
[InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
unsafe interface IMemoryBufferByteAccess
{
    void GetBuffer(out byte* buffer, out uint capacity);
}

And of course, your project will have to allow unsafe code for all of this to work.

Finally, this is how you can save a SoftwareBitmap to JPEG file:

private static async Task SaveSoftwareBitmapAsync(SoftwareBitmap bitmap, StorageFile file)
{
    using (var outputStream = await file.OpenAsync(FileAccessMode.ReadWrite))
    {
        var encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.JpegEncoderId, outputStream);

        // Grab the data from the SoftwareBitmap
        encoder.SetSoftwareBitmap(bitmap);
        await encoder.FlushAsync();
    }
}