1

I am trying to find an efficient way to see if one image is a subset of another (meaning that each unique pixel in one image is also found in the other.) The repetition or ordering of the pixels do not matter.

I am working in Java, so I would like all of my operations to be completed in OpenCV for efficiency's sake.

My first idea was to export a list of unique pixel values, and compare it to the list from the second image. As there is not a built in function to extract unique pixels, I abandoned this approach.

I also understand that I can find the locations of a particular color with the inclusive inRange, and findNonZero operations. 

  Core.inRange(image, color, color, tempMat); // inclusive
  Core.findNonZero(tempMat, colorLocations);

Unfortunately, this does not provide an adequate answer, as it would need to be executed per color, and would still require extracting unique pixels.

Essentially, I'm asking if there is a clever way to use the built in OpenCV functions to see if an image is comprised of the pixels found in another image.

I understand that this will not work for slight color differences. I am working on a limited dataset, and care about the exact pixel values.

To put the question more mathematically:

CA2C7B
  • 368
  • 1
  • 14

2 Answers2

3

Because the only think you are interested in is the pixel values i would suggest to do the following.

  1. Compute the histogram of image 1 using hist1 = calcHist()
  2. Compute the histogram of image 2 using hist2 = calcHist()
  3. Calculate the difference vector diff = hist1 - hist2
  4. Check if each bin of the hist of the subimage is less or equal than the corresponding bin in the hist of the bigger image

Thanks to Miki for the fix.

Amitay Nachmani
  • 3,259
  • 1
  • 18
  • 21
  • This doesn't work. If you have, say, completely blue images, then both the original image and the subset image have a single bin histogram, but the height of the bins is very different. The difference of the histogram is non zero, but the answer should be true – Miki Jul 24 '17 at 22:20
  • @Miki, I need to look into OpenCV histograms more, but I believe using the compareHist function with the CV_COMP_INTERSECT method would work. – CA2C7B Jul 24 '17 at 22:32
  • This works if you already know the position of the subimage in the bigger image and you just need to check if they have the same amount of pixels of the same color. Otherwise, as soon as you start counting pixels that are outside of the subimage, this algorithm is flawed @gabe870 – Miki Jul 24 '17 at 22:36
  • @Miki, I don't care about the order or count of the pixels. I simply want to see if the set of pixel values in one image is a subset of the pixel values in another. (pixel values meaning RGB, not location.) – CA2C7B Jul 24 '17 at 22:39
  • Ok then.... But this is not what it's stated in your question. However, the algorithm is still flawed. Instead of checking for "zero-difference", you just need to check if each bin of the hist of the subimage is less or equal than the corresponding bin in the hist of the bigger image @gabe870 – Miki Jul 24 '17 at 22:43
  • @Miki, I believe I've asked that. I said that the repetition and ordering of the pixels do not matter. I understand it it somewhat of a strange request. I'm going to accept this answer as it has definitely set me down the correct path. I'll come back tomorrow to report the exact method used. – CA2C7B Jul 24 '17 at 22:48
  • @Amitay step 4 is simply: check that you don't have any negative value in the diff (assuming image1 is the bigger image). – Miki Jul 24 '17 at 22:50
3

I will keep Amitay's as the accepted answer, as he absolutely lead me down the correct path. I wanted to also share my exact answer for anyone who finds this in the future.

As I stated in my question, I was looking for an efficient way to see if the RGB values of one image were a subset of the RGB values of another image.

I made a function to the following specification:

The Java code is as follows: 

private boolean isSubset(Mat subset, Mat subMask, Mat superset) {
    // Get unique set of pixels from both images
    subset = getUniquePixels(subset, subMask);
    superset = getUniquePixels(superset, null);


    // See if the superset pixels encapsulate the subset pixels
    // OR the unique pixels together
    Mat subOrSuper = new Mat();
    Core.bitwise_or(subset, superset, subOrSuper);

    //See if the ORed matrix is equal to the superset
    Mat notEqualMat = new Mat();
    Core.compare(superset, subOrSuper, notEqualMat, Core.CMP_NE);
    return Core.countNonZero(notEqualMat) == 0;
}

subset and superset are assumed to be CV_8UC3 matricies, while subMask is assumed to be CV_8UC1. 

private Mat getUniquePixels(Mat img, Mat mask) {
    if (mask == null) {
        mask = new Mat();
    }

    // int bgrValue = (b << 16) + (g << 8) + r;
    img.convertTo(img, CvType.CV_32FC3);
    Vector<Mat> splitImg = new Vector<>();
    Core.split(img, splitImg);
    Mat flatImg = Mat.zeros(img.rows(), img.cols(), CvType.CV_32FC1);
    Mat multiplier;

    for (int i = 0; i < splitImg.size(); i++) {
        multiplier = Mat.ones(img.rows(), img.cols(), CvType.CV_32FC1);
        // set powTwo = to 2^i;
        int powTwo = (1 << i);
        // Set multiplier matrix equal to powTwo;
        Core.multiply(multiplier, new Scalar(powTwo), multiplier);

        // n<<i == n * 2^i;
        // I'm shifting the RGB values into separate parts of the same 32bit
        // integer.
        Core.multiply(multiplier, splitImg.get(i), splitImg.get(i));

        // Add the shifted RGB components together.
        Core.add(flatImg, splitImg.get(i), flatImg);
    }

    // Create a histogram of the pixel values.
    List<Mat> images = new ArrayList<>();
    images.add(flatImg);
    MatOfInt channels = new MatOfInt(0);
    Mat hist = new Mat();
    // 16777216 == 256*256*256
    MatOfInt histSize = new MatOfInt(16777216);
    MatOfFloat ranges = new MatOfFloat(0f, 16777216f);

    Imgproc.calcHist(images, channels, mask, hist, histSize, ranges);
    Mat uniquePixels = new Mat();
    Core.inRange(hist, new Scalar(1), new Scalar(Float.MAX_VALUE), uniquePixels);

    return uniquePixels;
}

Please feel free to ask questions, or point out problems!

CA2C7B
  • 368
  • 1
  • 14