Send InkCanvas signature using WCF call

Question

I have a Windows 10 UWP app that will run on Windows 10 Mobile. A requirement I have is to capture a signature from the user. So far, I am simply using an InkCanvas in XAML and have it wired up to my code behind.

I then have a button that when clicked, will take the signature on the InkCanvas and send it to the server via a WCF call. The server and WCF service is already existing. It takes in the signature image as a base64 serialized string.

I know how to get the base64 once I have either an image or a byte array. However, in my many hours of reading, I am finding that articles/examples were either written for WPF or Windows 8.1 and do not work on Windows 10 UWP. Also, of the examples that I have found that will work, it seems my only option is to save the signature to file as a GIF.

I see that I can call GetStrokes() like this

 var strokeCollection = cvsSignature.InkPresenter.StrokeContainer.GetStrokes();

Which will return me a read only list of InkStroke. I guess I can iterate that list and build a byte array? How would I do that? It seems this is not efficient?

Otherwise, I thought I could just change the stream from a file stream to a memory stream but I guess either this is not possible or I am missing something. I am trying this

 using (var inkMemStream = new MemoryStream())
 {
     await cvsSignature.InkPresenter.StrokeContainer.SaveAsync(inkMemStream);
 }

But with this type of approach I get an exception that I cannot convert from System.IO.MemoryStream to Windows.Storage.Streams.IOutputStream

Thanks!

Answer 1

I found a way, if you don't want to save your InkCanvas to a file or as a GIF, to manipulate it in memory. I actually have three options below. Note, for some of this, you will need to add a reference to Win2D.uwp, which can be found on NuGet by searching that exact name. It is provided by Microsoft.

1. Convert the InkCanvas to a byte array:

   private byte[] ConvertInkCanvasToByteArray()
   {
       //First, we need to get all of the strokes in the canvas
       var canvasStrokes = myCanvas.InkPresenter.StrokeContainer.GetStrokes();
   
       //Just as a check, make sure to only do work if there are actually strokes (ie not empty)
       if (canvasStrokes.Count > 0)
       {
           var width = (int)myCanvas.ActualWidth;
           var height = (int)myCanvas.ActualHeight;
           var device = CanvasDevice.GetSharedDevice();
           //Create a new renderTarget with the same width and height as myCanvas at 96dpi
           var renderTarget = new CanvasRenderTarget(device, width,
               height, 96);
   
           using (var ds = renderTarget.CreateDrawingSession())
           {
               //This will clear the renderTarget with a clean slate of a white background.
               ds.Clear(Windows.UI.Colors.White);
               //Here is where we actually take the strokes from the canvas and draw them on the render target.
               ds.DrawInk(myCanvas.InkPresenter.StrokeContainer.GetStrokes());
           }
           //Finally, this will return the render target as a byte array.
           return renderTarget.GetPixelBytes();
       }
       else
       {
           return null;
       }
   }
   

2. If all you need is a byte array of the pixels, you are done. However, if you now want to generate an image, you use the WriteableBitmap to do so. Here are sync and async methods that can be called to do this.

   private WriteableBitmap GetSignatureBitmapFull()
   {
       var bytes = ConvertInkCanvasToByteArray();
   
       if (bytes != null)
       {
           var width = (int)cvsSignature.ActualWidth;
           var height = (int)cvsSignature.ActualHeight;
   
           var bmp = new WriteableBitmap(width, height);
           using (var stream = bmp.PixelBuffer.AsStream())
           {
               stream.Write(bytes, 0, bytes.Length);
               return bmp;
           }
       }
       else
           return null;
   }
   
   private async Task<WriteableBitmap> GetSignatureBitmapFullAsync()
   {
       var bytes = ConvertInkCanvasToByteArray();
   
       if (bytes != null)
       {
           var width = (int)cvsSignature.ActualWidth;
           var height = (int)cvsSignature.ActualHeight;
   
           var bmp = new WriteableBitmap(width, height);
           using (var stream = bmp.PixelBuffer.AsStream())
           {
               await stream.WriteAsync(bytes, 0, bytes.Length);
               return bmp;
           }
       }
       else
           return null;
   }
   

3. Finally, here is an async method that I am using to be able to do a crop on the bitmap if you want to do that. The key thing with this method is notice how with this approach, you don't have to get the pixel bytes as an array first. You just get the strokes from the canvas and then it is saved directly into a memory stream

   private async Task<WriteableBitmap> GetSignatureBitmapCropped()
   {
       try
       {
           var canvasStrokes = cvsSignature.InkPresenter.StrokeContainer.GetStrokes();
   
           if (canvasStrokes.Count > 0)
           {
               var bounds = cvsSignature.InkPresenter.StrokeContainer.BoundingRect;
               var xOffset = (uint)Math.Round(bounds.X);
               var yOffset = (uint)Math.Round(bounds.Y);
               var pixelWidth = (int)Math.Round(bounds.Width);
               var pixelHeight = (int)Math.Round(bounds.Height);
   
               using (var memStream = new InMemoryRandomAccessStream())
               {
                   await cvsSignature.InkPresenter.StrokeContainer.SaveAsync(memStream);
   
                   var decoder = await BitmapDecoder.CreateAsync(memStream);
   
                   var transform = new BitmapTransform();
                   var newBounds = new BitmapBounds();
                   newBounds.X = 0;
                   newBounds.Y = 0;
                   newBounds.Width = (uint)pixelWidth;
                   newBounds.Height = (uint)pixelHeight;
                   transform.Bounds = newBounds;
   
                   var pdp = await decoder.GetPixelDataAsync(
                       BitmapPixelFormat.Bgra8,
                       BitmapAlphaMode.Straight,
                       transform,
                       ExifOrientationMode.IgnoreExifOrientation,
                       ColorManagementMode.DoNotColorManage);
   
                   var pixels = pdp.DetachPixelData();
   
                   var cropBmp = new WriteableBitmap(pixelWidth, pixelHeight);
   
                   using (var stream = cropBmp.PixelBuffer.AsStream())
                   {
                       await stream.WriteAsync(pixels, 0, pixels.Length);
                   }
                   return cropBmp;
               }
           }
           else
           {
               return null;
           }
       }
       catch (Exception ex)
       {
           return null;
       }
   }
   

Hope this helps provide some alternatives when using InkCanvas in Windows 10 Universal.

Answer 2

it seems my only option is to save the signature to file as a GIF.

GIF is not the only choice. The official sample just show GIF, but you can also save the InkStokes collection to JPG and PNG. Code as follows:

 var savePicker = new Windows.Storage.Pickers.FileSavePicker();
 savePicker.SuggestedStartLocation = Windows.Storage.Pickers.PickerLocationId.PicturesLibrary;
 savePicker.FileTypeChoices.Add("Gif,JPG,PNG", new System.Collections.Generic.List<string> { ".jpg" ,".gif",".png"}); 
 Windows.Storage.StorageFile file = await savePicker.PickSaveFileAsync();

But with this type of approach I get an exception that I cannot convert from System.IO.MemoryStream to Windows.Storage.Streams.IOutputStream

As you see, it cannot convert from System.IO.MemoryStream to Windows.Storage.Streams.IOutputStream directly. You need to read the image file you just saved as ImageSource. As you known how to get the base64 once you have either an image or a byte array. So what we just need is to get an image or a byte array from the image file.

Read the image file as byte array and then a memory stream as follows

if (inkCanvas.InkPresenter.StrokeContainer.GetStrokes().Count > 0)
{
    var savePicker = new Windows.Storage.Pickers.FileSavePicker();
    savePicker.SuggestedStartLocation = Windows.Storage.Pickers.PickerLocationId.PicturesLibrary;
    savePicker.FileTypeChoices.Add("Gif,JPG,PNG", new System.Collections.Generic.List<string> { ".jpg", ".gif", ".png" });
    Windows.Storage.StorageFile file = await savePicker.PickSaveFileAsync();
    if (null != file)
    {
        using (IRandomAccessStream stream = await file.OpenAsync(FileAccessMode.ReadWrite))
        {
            await inkCanvas.InkPresenter.StrokeContainer.SaveAsync(stream);
        }
        using (IRandomAccessStream streamforread = await file.OpenAsync(FileAccessMode.Read))
        {
            WriteableBitmap inkimagesource = new WriteableBitmap(50, 50);
            inkimagesource.SetSource(streamforread);
            byte[] imageBuffer = inkimagesource.PixelBuffer.ToArray();
            MemoryStream ms = new MemoryStream(imageBuffer);
        }
    }
}

Read the file as image source just need to change the file reading code as follows:

using (IRandomAccessStream streamforread = await file.OpenAsync(FileAccessMode.Read)
{ 
   var bitmap = new BitmapImage();
   bitmap.SetSource(streamforread); 
}

I guess I can iterate that list and build a byte array? How would I do that? It seems this is not efficient?

It seems like currently there is no API can build the InkStroke collections as byte array directly. SaveAsync() method has already help you save the InkStroke collection to a stream, you can use code above for using it.