OpenCV java alpha Blending

Question

I am trying to port the following code (from C++ to Java) to make good alpha blending between my images, but It did not work:

#include opencv2/opencv.hpp

using namespace cv;
using namespace std;

int main(int argc, char** argv)
{

    // Read the images
    Mat foreground = imread("puppets.png");
    Mat background = imread("ocean.png");
    Mat alpha = imread("puppets_alpha.png");

    // Convert Mat to float data type
    foreground.convertTo(foreground, CV_32FC3);
    background.convertTo(background, CV_32FC3);

    // Normalize the alpha mask to keep intensity between 0 and 1
    alpha.convertTo(alpha, CV_32FC3, 1.0/255); // 

    // Storage for output image
    Mat ouImage = Mat::zeros(foreground.size(), foreground.type());

    // Multiply the foreground with the alpha matte
    multiply(alpha, foreground, foreground); 

    // Multiply the background with ( 1 - alpha )
    multiply(Scalar::all(1.0)-alpha, background, background); 

    // Add the masked foreground and background.
    add(foreground, background, ouImage); 

    // Display image
    imshow("alpha blended image", ouImage/255);
    waitKey(0);

    return 0;
}

The code can be found here: https://www.learnopencv.com/alpha-blending-using-opencv-cpp-python/ And my Java version:

public static Mat alphaBlend(Mat background, Mat foreground) {
        Vector<Mat> rgba = new Vector<Mat>();
        // split RBGA image for separate channels
        Core.split(background, rgba);
        // get alpha channel
        Mat alpha = rgba.get(3);

        // Convert Mat to float data type
        foreground.convertTo(foreground, CvType.CV_32FC3);
        background.convertTo(background, CvType.CV_32FC3);

        // Normalize the alpha mask to keep intensity between 0 and 1
        alpha.convertTo(alpha, CvType.CV_32FC3, 1.0/255); //
        Imgproc.cvtColor(alpha,alpha, Imgproc.COLOR_GRAY2BGRA,4);

        Mat outImage = Mat.zeros(foreground.size(),foreground.type());

        // Multiply the foreground with the alpha matte
        Core.multiply(alpha, foreground, foreground);

        Mat kernel = new MatOfDouble(1.0);

        Core.subtract(kernel, alpha, alpha);

        // Multiply the background with ( 1 - alpha )
        Core.multiply(alpha, background, background);

        // Add the masked foreground and background.
        Core.add(foreground, background, outImage);

        Core.divide(new MatOfDouble(255), outImage,outImage);
        return outImage;
    }

I think my problem is to port this in java:

multiply(Scalar::all(1.0)-alpha, background, background);

Any help would be appreciated!

Thanks

Edit: Here a working version but the border does not blend transparent pixel causing blank pixel/line at the border of the intersection of the images:

    private static Mat merge(Mat background, Mat foreground) {
        Vector<Mat> rgba = new Vector<Mat>();
        // split RBGA image for separate channels
        Core.split(background, rgba);
        // get alpha channel
        Mat alpha = rgba.get(3);
        // Convert Mat to float data type
        // Normalize the alpha mask to keep intensity between 0 and 1
        alpha.convertTo(alpha, CvType.CV_32FC3, 1.0/255); //
        Mat dst = new Mat(256,256,CvType.CV_8UC4);
        alpha.convertTo(alpha,CvType.CV_8UC1);
        Mat alphaInv = new Mat();
        // invert the mask
        Core.absdiff(new MatOfDouble(1.0), alpha, alphaInv);
        foreground.copyTo(dst, alphaInv);
        // case where foreground is full JPEG in BGR
        if(dst.type() != CvType.CV_8UC4) {
            Imgproc.cvtColor(dst, dst, Imgproc.COLOR_BGR2BGRA, 4);
        }
        background.copyTo(dst, alpha);
        return dst;
    }

score 0 · Answer 1 · answered Dec 22 '19 at 05:27

0

I recommend using the addWeighted method from Core for basic alpha-blending.

It looks like you want a variable alpha blend based on a mask, you will need to arrange your parameters in the order specified by the api for basic things like addition/subtraction/multiplication. For your multiplication this means the scalar should be the second parameter. For your subtraction this means the scalar should again be the second parameter (I find myself multiplying the matrix by negative 1 followed by adding to the scalar value).

answered Dec 22 '19 at 05:27

shortcipher3

1,292
9
22

The addWeighted function mixes both image giving transparency even for intersecting pixels. What I would like is to put an image to another one without transparency (no mixing color) for the intersecting part. The tutorial gives the good result (I tried in python and it is exactly the result I want). – nvsl Dec 22 '19 at 09:47
Did you try rearranging your argument order, following the API linked in the second paragraph? The java generally requires the scalar value to be the seconds argument, it looks like you are setting it as the first – shortcipher3 Dec 22 '19 at 14:16
Yes I did, I have tested a lot of combination without any success. I don't know what is wrong with my code – nvsl Dec 22 '19 at 15:20
I added another answer that I think may simplify what you are doing, try it out and let me know how it goes – shortcipher3 Dec 22 '19 at 15:56

score 0 · Answer 2 · answered Dec 22 '19 at 15:55

0

Based on the link you shared, it looks like the goal is just to use image masking, I think there is an easier way to do it, using the copyTo function. Try putting this in your main and when it is working move to your function:

// Read the images
Mat foreground = Imgcodecs.imread("puppets.png");
Mat background = Imgcodecs.imread("ocean.png");
Mat mask = Imgcodecs.imread("puppets_alpha.png");
Mat notMask = new Mat();
Core.bitwise_not(mask, notMask);
Mat result = new Mat();
Core.copyTo(background, result, mask);
Core.copyTo(foreground, result, notMask);
Imgcodecs.imwrite("result.png", result);

answered Dec 22 '19 at 15:55

shortcipher3

1,292
9
22

1

Thanks for you help. The code above does not deliver a good result but my first attempt was in that direction. Using the alpha band as a mask and copy pixel using this mask. It worked fine but the problem is for transparent pixel between 0...1 in the border. Because for that pixel, it does full white pixel. It is for that I am trying to blend these pixels to have smooth intersection – nvsl Dec 22 '19 at 16:09
You can see my edit to see a basic working function but still not perfect. Needed blending border transparent pixels – nvsl Dec 22 '19 at 16:16

score -1 · Answer 3 · answered Mar 14 '22 at 14:04

using OpenCVForUnity.CoreModule;
using OpenCVForUnity.UnityUtils;
using UnityEngine;
public class bitwiseNew : MonoBehaviour
{
    Texture2D textureFace, textureMask, textureBG;
    void Start()
    {
        Utils.setDebugMode(true);
        textureFace = Resources.Load("Lena") as Texture2D;
        textureMask = Resources.Load("circle") as Texture2D;
        textureBG = Resources.Load("face") as Texture2D;

        Mat matFace = new Mat(textureFace.height, textureFace.width, CvType.CV_8UC4);
        Mat matMask = new Mat(textureMask.height, textureMask.width, CvType.CV_8UC4);
        Mat matBG = new Mat(textureBG.height, textureBG.width, CvType.CV_8UC4);

        Utils.texture2DToMat(textureFace, matFace);
        Utils.texture2DToMat(textureMask, matMask);
        Utils.texture2DToMat(textureBG, matBG);

        Mat mask = new Mat(matFace.cols(), matFace.rows(), matFace.type());
        Mat maskDst1 = new Mat(matFace.cols(), matFace.rows(), matFace.type());
        Mat maskDst2 = new Mat(matFace.cols(), matFace.rows(), matFace.type());
        Mat dst = new Mat(matFace.cols(), matFace.rows(), matFace.type());

        Core.bitwise_not(matMask, mask);
        Core.bitwise_not(matFace, matFace);

        Core.bitwise_and(matBG, mask, maskDst1);
        Core.bitwise_or(mask, matFace, maskDst2);
        Core.bitwise_not(maskDst2, maskDst2);

        Core.addWeighted(maskDst1, 1f, maskDst2, 1f, 0f, dst);
        Debug.Log("dst: " + dst.channels());

        Texture2D texture = new Texture2D(dst.cols(), dst.rows(), TextureFormat.RGBA32, false);
        Utils.matToTexture2D(dst, texture);
        gameObject.GetComponent<Renderer>().material.mainTexture = texture;
        Utils.setDebugMode(false);
    }
}

[A code-only answer is not high quality](//meta.stackoverflow.com/questions/392712/explaining-entirely-code-based-answers). While this code may be useful, you can improve it by saying why it works, how it works, when it should be used, and what its limitations are. Please [edit] your answer to include explanation and link to relevant documentation. — Muhammad Mohsin Khan, Mar 14 '22 at 14:38

OpenCV java alpha Blending

3 Answers3