Mask out color in Image canvas flutter

Question

I am trying to add skin tone to an image in flutter canvas.

I've used the following code before to apply chroma by altering pixels when I load the image:

  static Future<ui.Image> applyChromaToImage(ui.Image image, String pathForImage, RGBPixel chromaToApply, {RGBPixel previousChromaColor}) async
{
    List<ChromaPointRange> chromaPoints = processChromaImageBytes(
          image, imgBytes);
    for(ChromaPointRange rnge in chromaPoints)
    {
      for(int y = rnge.yValStart; y <= rnge.yValEnd; y++)
      {
        RGBPixel currentPixel = RGBPixel.generatePixelFromImagePos(imgBytes, image.width, rnge.xVal, y);
        //replace current pixel with skin tone
        RGBPixel newPixl = currentPixel.mergeSkinTonePixel(chromaToApply, previousChromaColor);
        imgBytes.setUint32((y * image.width + rnge.xVal) * 4, newPixl.getHex());
      }
    }

    final Completer<ui.Image> imageCompleter = new Completer();

    //looks the endian format doesn't get set right here
    ui.PixelFormat formatToUse = Endian.host == Endian.little ? ui.PixelFormat.rgba8888 : ui.PixelFormat.bgra8888;


    ui.decodeImageFromPixels(
      imgBytes.buffer.asUint8List(),
      image.width,
      image.height,
      formatToUse,
          (ui.Image result) {
        imageCompleter.complete(result);
        // use your result image
      },
    );


    //return image;
    return await imageCompleter.future;
}

static List<ChromaPointRange> processChromaImageBytes(ui.Image image, ByteData imgBytes)
  {
    List<ChromaPointRange> chromaPoints = [];

    ChromaPointRange currentPoints = null;
    for(int x = 0; x < image.width; x = x + 1)
    {
      for(int y = 0; y < image.height; y = y + 1)
      {
        RGBPixel currentPixel = RGBPixel.generatePixelFromImagePos(imgBytes, image.width, x, y);
        if(currentPixel.isSkinTonePixel())
        {
          if(currentPoints == null)
          {
            currentPoints = ChromaPointRange.fromEmpty();
            currentPoints.xVal = x;
            currentPoints.yValStart = y;
          }
        }
        else if(currentPoints != null)
        {
          currentPoints.yValEnd = y - 1;
          chromaPoints.add(currentPoints);
          currentPoints = null;
        }
      }

      if(currentPoints != null)
      {
        currentPoints.yValEnd = image.height - 1;
        chromaPoints.add(currentPoints);
        currentPoints = null;
      }
    }

    return chromaPoints;

  }

which basically checks every pixel in the image to see if it's within a range of the target color ( with is RGB 0, 255, 0), then adjusts the pixel if it is. This works, but takes a really long time ~ 3 seconds for a 1920 x 1080 image.

The end result is that I want to paint the image to a canvas with a skin tone applied. I've tried a different strategy, by painting the color underneath the image, and then trying to mask out that color from the image using canvas color filters. This is 1000% faster, but doesn't quite work.

Here is the code:

renderSprite(Canvas canvasToRender, Offset offsetToRender)
  {
    Paint imgPaint = new Paint();
    if(chromaToApply != null)
    {
      Paint chromaPaint = new Paint();
      chromaPaint.colorFilter = ColorFilter.mode(Color.fromRGBO(chromaToApply.redVal, chromaToApply.greenVal, chromaToApply.blueVal, 1), BlendMode.modulate);
      canvasToRender.drawImage(spriteImage, offsetToRender, chromaPaint);
      imgPaint.colorFilter = ColorFilter.mode(Color.fromRGBO(0, 255, 0, 1), BlendMode.dstOut);
    }

    if(spriteImage != null)
      canvasToRender.drawImage(spriteImage, offsetToRender, imgPaint);
  }

Here is the image that is painted underneath

Here is the image that is painted ontop

So I'm trying to mask out the green so the tan skin tone shows through on specific parts of the image.

I can't seem to find any combination of ColorFilter or anything else that will mask out the green color for me from the canvas. Any suggestions?