Drawing the Difference of two Images to Canvas

Question

I am trying to draw the difference of two Images I have to the Canvas in Flutter and what I really mean is that I only want to draw what is different (you can see what I expect to see below).
It needs to happen synchronously, i.e. in a paint call!

In this answer, the two images are referred to as imageA and imageB.

Code

canvas.drawImage(
  imageA,
  offset,
  Paint(),
);
canvas.drawImage(
  imageB,
  offset,
  Paint()
    ..blendMode = BlendMode.difference,
);

Visualization

^{imageA and imageB drawn separately.}

^{Difference between imageA and imageB. You can see what I expect to see left and what I actually get using the code above right.
The gray/white background color in the left image is supposed to be transparent, i.e. alpha should equal zero where the two images are the same. Additionally, the black shadow around the red square in the expected image is an artifact and not what I want in my app.}

Problem

I am not saying that the expected image is what I expect to get from BlendMode.difference, but I want to know how I can get the expected output, i.e. how I achieve only drawing what is different between two images.
This means that I want to only render pixels from imageB that are different from imageA and otherwise nothing, i.e. alpha value 0.

Explanation

I will try to explain it a bit more clearly again:

• If the color is the same, draw transparent pixel (remove source pixel).

• If the color is different, draw destination pixel.

Answer 1

Currently flutter does not support what you are trying to do. Based on my research, flutter handles canvas paint on the native level. You can see this in action from this snippet:

  /// Fills the canvas with the given [Paint].
  ///
  /// To fill the canvas with a solid color and blend mode, consider
  /// [drawColor] instead.
  void drawPaint(Paint paint) {
    assert(paint != null);
    _drawPaint(paint._objects, paint._data);
  }
  void _drawPaint(List<dynamic> paintObjects, ByteData paintData) native 'Canvas_drawPaint';

This is from the canvas implementation on the latest version of flutter (1.9.2). Any pixel manipulation that you can do without async code must be done utilizing flutter BlendMode or ImageFilter which lacks personalized masks support.

The only way to do what you intend to do is to actually implement native code, which I don't really think you want to do. I spent the last 6 hours trying to figure out if there wasn't anything out of the official flutter API docs that would do what you want it to do, but the thing is, most of things in flutter are just intended to be async.

If you are going to venture into implementing this natively, why don't you ask to be implemented as a feature in the official flutter github? I mean, so much work, don't let it be wasted.

Just because I didn't wanted to leave empty handed, I have written precisely what you wanted with the image lib but using async:

import 'dart:typed_data';
import 'dart:ui' as ui;
import 'package:image/image.dart' as im;

import 'package:flutter/material.dart';

void main() => runApp(App());

class App extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MyApp();
  }
}

class MyApp extends StatefulWidget {
  @override
  _MyAppState createState() => _MyAppState();
}

class _MyAppState extends State<MyApp> {
  ByteBuffer imgBytes;
  double canvasHeight = 500;
  double canvasWidth = 300;

  ui.PictureRecorder recorder1;
  Canvas canvas1;

  ui.PictureRecorder recorder2;
  Canvas canvas2;

  ui.Picture picture1;
  ui.Picture picture2;

  @override
  void initState() {
    // Canvas1
    recorder1 = ui.PictureRecorder();
    canvas1 = Canvas(recorder1,
        Rect.fromPoints(Offset(0.0, 0.0), Offset(canvasWidth, canvasHeight)));

    // Draw red background
    var paint = Paint();
    paint.style = PaintingStyle.fill;
    paint.color = Colors.red;

    canvas1.drawRect(Offset(0, 0) & Size(canvasWidth, canvasHeight), paint);

    // Draw black square
    paint = Paint();
    paint.style = PaintingStyle.fill;
    paint.color = Colors.black;

    canvas1.drawRect(
        Offset(canvasWidth * 0.5 / 4, canvasHeight / 2) &
            Size(canvasHeight / 8, canvasHeight / 8),
        paint);

    picture1 = recorder1.endRecording();

    // Canvas2
    recorder2 = ui.PictureRecorder();
    canvas2 = Canvas(recorder2,
        Rect.fromPoints(Offset(0.0, 0.0), Offset(canvasWidth, canvasHeight)));

    // Draw red background
    paint = Paint();
    paint.style = PaintingStyle.fill;
    paint.color = Colors.red;

    canvas2.drawRect(Offset(0, 0) & Size(canvasWidth, canvasHeight), paint);

    // Draw blue square
    paint = Paint();
    paint.style = PaintingStyle.fill;
    paint.color = Colors.blue;

    canvas2.drawRect(
        Offset(canvasWidth * 2.5 / 4, canvasHeight / 2) &
            Size(canvasHeight / 8, canvasHeight / 8),
        paint);

    picture2 = recorder2.endRecording();

    (() async {
      ui.Image img1 =
          await picture1.toImage(canvasWidth.toInt(), canvasHeight.toInt());
      ByteData byteData1 =
          await img1.toByteData(format: ui.ImageByteFormat.png);
      im.Image decodedPng1 = im.decodePng(byteData1.buffer.asUint8List());

      ui.Image img2 =
          await picture2.toImage(canvasWidth.toInt(), canvasHeight.toInt());
      ByteData byteData2 =
          await img2.toByteData(format: ui.ImageByteFormat.png);
      im.Image decodedPng2 = im.decodePng(byteData2.buffer.asUint8List());

      for (int i = 0; i < canvasHeight; i += 1) {
        for (int j = 0; j < canvasWidth; j += 1) {
          int pixel1 = decodedPng1.getPixel(j, i);
          int r1 = pixel1 & 0xff;
          int g1 = (pixel1 >> 8) & 0xff;
          int b1 = (pixel1 >> 16) & 0xff;
          int a1 = (pixel1 >> 24) & 0xff;

          int pixel2 = decodedPng2.getPixel(j, i);
          int r2 = pixel2 & 0xff;
          int g2 = (pixel2 >> 8) & 0xff;
          int b2 = (pixel2 >> 16) & 0xff;
          int a2 = (pixel2 >> 24) & 0xff;

          if (r1 == r2 && g1 == g2 && b1 == b2 && a1 == a2) {
            // Draw black transparent
            decodedPng2.setPixel(j, i, 0);
          } else {
            // Leave as is
          }
        }
      }

      setState(() {
        imgBytes = Uint8List.fromList(im.encodePng(decodedPng2)).buffer;
      });
    })();
  }

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: Scaffold(
        appBar: AppBar(
          title: Text('Dummy'),
        ),
        body: imgBytes != null
            ? Center(
                child: Image.memory(
                  Uint8List.view(imgBytes),
                  width: canvasWidth,
                  height: canvasHeight,
                ),
              )
            : Container(),
      ),
    );
  }
}