Extract the floor layout and threshold with OpenCV and Python

Question

I've tried to use SSIM to extract the difference between two images to get only the floor area (image_a is the original and image_b has painted floor).

The output that was expected, is a threshold mask.

The problem I had was that the thresholding of ssim difference just didn't work in my case (example is shown below).

Can someone provide a better technique or theory of thresholding?

from skimage.measure import compare_ssim
import cv2
...

image_a = cv2.imread(first)
image_b = cv2.imread(second)

gray_a = cv2.cvtColor(image_a, cv2.COLOR_BGR2GRAY)
gray_b = cv2.cvtColor(image_b, cv2.COLOR_BGR2GRAY)

_, diff = compare_ssim(gray_a, gray_b, full=True, gaussian_weights=True)
diff = (diff * 255).astype("uint8")

thresh = cv2.threshold(diff, 0, 255,
                       cv2.THRESH_BINARY_INV | cv2.THRESH_OTSU)[1]

contours = cv2.findContours(thresh.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]

contour_sizes = [(cv2.contourArea(contour), contour) for contour in contours]


if len(contour_sizes) > 0:
    largest_contour = max(contour_sizes, key=lambda x: x[0])[1]
    x, y, w, h = cv2.boundingRect(largest_contour)
    cv2.rectangle(image_a, (x, y), (x + w, y + h), (36, 255, 12), 2)
    cv2.rectangle(image_b, (x, y), (x + w, y + h), (36, 255, 12), 2)

cv2.imwrite('image_a.jpg', image_a)
cv2.imwrite('image_b.jpg',image_b)
cv2.imwrite('thresh.jpg', thresh)

image_a with max contour detected image_b with max contour detected thresh

Try erode and dilate. [This article](https://docs.opencv.org/3.4/db/df6/tutorial_erosion_dilatation.html) might give you some idea. Simply put, tune eroding and dilating parameters till you're satisfied with the result and then use `findContours` method to get floor area. — Cabara, Oct 04 '19 at 07:03
@陳翰群 Thanks, It's a helpful article, but it removes not all artifacts that I have. What about a better diff extraction algorithm? — arturkuchynski, Oct 04 '19 at 07:56
Furthermore, [contour features](https://docs.opencv.org/trunk/dd/d49/tutorial_py_contour_features.html) could help you filter out parts you don't want. For example, you may be looking for contours with larger areas. In your case, you want to find the bottom part and the bottom-right corner. These two parts has significantly larger areas than other artifacts. — Cabara, Oct 04 '19 at 09:04
Hey @arturkuchynski, I'm working on a similar project, can you please tell me the process or at least, point a direction to achieve the masking of the floor as in second image from the first image. Thanks. I've researched a lot about it, but couldn't find which direction to head to. PyImageSearch's OpenCV blogs are informative but don't feel like the solution I need. — Lalit Fauzdar, Jun 14 '21 at 13:50

arturkuchynski · Accepted Answer · 2019-10-07T05:54:19.883

A better result can be obtained by thresholding the mean of the difference beetween given images.

def get_mask(img1, img2, thresh):
    if img1.shape != img2.shape:
        return
    diff = cv2.absdiff(img1, img2)
    diff = np.mean(diff, axis=2)
    diff[diff <= thresh] = 0
    diff[diff > thresh] = 255
    mask = np.dstack([diff] * 3)
    return mask

Artifacts may appear in the resulting mask and can be reduced by applying Morphological Transformations.

Extract the floor layout and threshold with OpenCV and Python

1 Answers1