How to extract a specific section of an image using OpenCV in Python?

Question

I am trying to extract a portion of an image by performing Canny edge detection. I have succesfully created a mask of that object. But when I perform a bitwise_and operation with the original image, to extract the foreground section, I am getting the following error .

OpenCV Error: Assertion failed ((mtype == CV_8U || mtype == CV_8S) && _mask.sameSize(*psrc1)) in cv::binary_op, file C:\projects\opencv-python\opencv\modules\core\src\arithm.cpp, line 241
    Traceback (most recent call last):
      File "C:\Users\Boudhayan Dev\Desktop\extraction.py", line 37, in <module>
        new_image = cv2.bitwise_and(img_rgb,img_rgb,mask=mask)
    cv2.error: C:\projects\opencv-python\opencv\modules\core\src\arithm.cpp:241: error: (-215) (mtype == CV_8U || mtype == CV_8S) && _mask.sameSize(*psrc1) in function cv::binary_op

My code is as follows -

import cv2
import numpy as np

img_rgb = cv2.imread("3.jpg")
cv2.namedWindow("Original Image",cv2.WINDOW_NORMAL)

img = cv2.cvtColor(img_rgb,cv2.COLOR_RGB2HSV)
img = cv2.bilateralFilter(img,9,105,105)
r,g,b=cv2.split(img)
equalize1= cv2.equalizeHist(r)
equalize2= cv2.equalizeHist(g)
equalize3= cv2.equalizeHist(b)
equalize=cv2.merge((r,g,b))

equalize = cv2.cvtColor(equalize,cv2.COLOR_RGB2GRAY)

ret,thresh_image = cv2.threshold(equalize,0,255,cv2.THRESH_OTSU+cv2.THRESH_BINARY)
equalize= cv2.equalizeHist(thresh_image)


canny_image = cv2.Canny(equalize,250,255)
canny_image = cv2.convertScaleAbs(canny_image)
kernel = np.ones((3,3), np.uint8)
dilated_image = cv2.dilate(canny_image,kernel,iterations=1)


new,contours, hierarchy = cv2.findContours(dilated_image, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
contours= sorted(contours, key = cv2.contourArea, reverse = True)[:10]
c=contours[0]
print(cv2.contourArea(c))
final = cv2.drawContours(img, [c], -1, (255,0, 0), 3)



mask = np.zeros(img_rgb.shape,np.uint8)
new_image = cv2.drawContours(mask,[c],0,255,-1,)
new_image = cv2.bitwise_and(img_rgb,img_rgb,mask=mask)


cv2.namedWindow("new",cv2.WINDOW_NORMAL)
cv2.imshow("new",new_image)

cv2.imshow("Original Image",img)
cv2.waitKey()

NOTE :- The code works fine if I try to perform the bitwise_and with a grayscale version of the image. However RGB,HSV or any other color spaces give the error above.

Please help.

EDIT 1 - The image in question is this -

EDIT 2-

The following is the result after using Numpy method. As you can see, the extracted image is the same size as the orange but it does not contain the orange instead the mask itself.

EDIT 3- @DanMašek and @lightalchemist , I could finally extract any foreground image.

Thank you

Please, provide a sample input image (e.g. that "3.jpg" your code refers to). Also, do some debugging -- what's the shape and data type of elements of the two inputs to `bitwise_and`? — Dan Mašek, Apr 20 '18 at 17:25
Can you visualize your mask? Note that you would probably have to convert the numpy array's type to uint8 before you can visualize it properly. — lightalchemist, Apr 20 '18 at 17:44
The problem is that your `mask` has 3 channels, but the documentation of [`cv2.bitwise_and`](https://docs.opencv.org/3.4.1/d2/de8/group__core__array.html#ga60b4d04b251ba5eb1392c34425497e14) explicitly states that "mask, 8-bit single channel array"... — Dan Mašek, Apr 20 '18 at 17:44
And is your result obtained using your original code or using my suggested solution below? — lightalchemist, Apr 20 '18 at 17:45
So, simply make the `mask` a 1 channel image: `mask = np.zeros(img_rgb.shape[:2], np.uint8)` — Dan Mašek, Apr 20 '18 at 17:46
@DanMašek - Thank you. I added the [:2] as you suggested and it worked. — Boudhayan Dev, Apr 20 '18 at 17:48
@lightalchemist- EDIT 2 is meant for your solution. It did not solve the problem entirely and that is because it was my fault. I agree on that. But I could extract the mask using your method. But thank you for your approach. — Boudhayan Dev, Apr 20 '18 at 17:49

Jeru Luke · Accepted Answer · 2018-04-20T18:48:49.293

4

I used the code provided above but only altered the line where the cv2.bitwise_and() is used:

new_image = cv2.bitwise_and(img_rgb, img_rgb, mask = equalize)

This is what I got and what you expected (I guess):

EDIT

I get it, you want to mask your image with an image of contour having the greatest area. In the following additional snippet I have binarized the image containing the contour of greatest area to be used as mask.

new_image = cv2.drawContours(mask,[c], -1, (255,255,255), -1)
new_image_gray = cv2.cvtColor(new_image, cv2.COLOR_BGR2GRAY)
ret, thresh1 = cv2.threshold(new_image_gray, 100, 255, cv2.THRESH_BINARY)
final = cv2.bitwise_and(img_rgb, img_rgb, mask = thresh1)

This is what I got:

Compared to the image above you do not see those holes inside the object of interest.

edited Apr 20 '18 at 18:48

answered Apr 20 '18 at 18:16

Jeru Luke

20,118
13
80
87

It works for this specific image. For other images, there is trace of unwanted background image here and there. That happens because the ` equalize ` variable stores edges besides the object edge as well. It is only after eliminating them via finding largest contour, that we end up with foreground image edge only. And that's why equalize is not used as mask in my code. But thanks anyway. – Boudhayan Dev Apr 20 '18 at 18:20
@BoudhayanDev I have edited the answer. Hope it suits your expectation. – Jeru Luke Apr 20 '18 at 18:49

score 1 · Answer 2 · answered Apr 20 '18 at 17:25

The error is telling you that you are trying to perform the bitwise_and operation on matrices where the entries are not integers. I believe the matrices also have to have the same number of channels. This is why it works with your grayscale image but not with the HSV image.

Instead of using bitwise_and, it is easier and more flexible to just use numpy matrix vectorization to perform the masking like so:

mask = np.zeros_like(img_rgb, dtype=np.uint8)

# I believe you want to draw the filled in contour on the mask
# You code actually assigns the resulting mask to new_image
# But that does not affect things as drawContours modifies mask in place
mask = cv2.drawContours(mask, [c] ,0, 255, -1) 

new_image = img_rgb.copy()
new_image[mask < 255] = 0  # Set values not masked to be 0

Note that if your mask is a single channel matrix instead of a 3 channel matrix, you would have to modify the code to be

new_image[mask < 255, :] = 0

I think the values are not integers mostly because of the equalization step performed earlier. Your code work, but the problem is the new image has the mask as the output. Instead of the mask I wanted the foreground image. I'll edit the answer to explain what is happening. — Boudhayan Dev, Apr 20 '18 at 17:33

How to extract a specific section of an image using OpenCV in Python?

2 Answers2