Bounding box on objects based on color python

Question

I try to draw a bounding box on every object in this picture, i wrote this code from documentation

import cv2 as cv2
import os
import numpy as np


img = cv2.imread('1 (2).png')
img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY);
ret,thresh = cv2.threshold(img,127,255,0)
im2,contours,hierarchy = cv2.findContours(thresh, 1, 2)
for item in range(len(contours)):
    cnt = contours[item]
    if len(cnt)>20:
        print(len(cnt))
        M = cv2.moments(cnt)
        cx = int(M['m10']/M['m00'])
        cy = int(M['m01']/M['m00'])
        x,y,w,h = cv2.boundingRect(cnt)
        cv2.rectangle(img,(x,y),(x+w,y+h),(0,255,0),2)
        cv2.imshow('image',img)
        cv2.waitKey(0)
        cv2.destroyAllWindows()

the result is only one object,

when i change the value 127 in this line to 200 in this line ret,thresh = cv2.threshold(img,127,255,0) i got different object.

here's the original image

The question is how can i detect all objects once?

first step is to read the manual: https://docs.opencv.org/2.4/modules/imgproc/doc/structural_analysis_and_shape_descriptors.html?highlight=findcontours#findcontours — Piglet, Apr 26 '18 at 21:34
Since you are interested in colour (or hue to be specific), it seems rather counterproductive to discard all the colour information by converting the BGR image to grayscale. Perhaps using the hue component from HSV or HSL would work better? — Dan Mašek, Apr 26 '18 at 22:04
@ZeyadEtman I'll give you the +1, but please [edit] your question, and add the necessary `import` statements to your sample script, so that someone else can simply copy and paste it and then run it without needs to make any significant modifications (other than, say, changing the filename). Otherwise well done on providing the necessary information. :) — Dan Mašek, Apr 26 '18 at 22:33
@ZeyadEtman Great. Just reading through the code, I noticed the following -- in the calls to `cv2.threshold` and `cv2.findContours` you use "magic numbers" for some of the parameters. In first case the `0` should be `cv2.THRESH_BINARY` and in second, the `1` a `cv2.RETR_LIST` and the `2` a `cv2.CHAIN_APPROX_SIMPLE`. It's always better to use the named constants for those parameters -- I hope you see how that makes the code easier to understand. — Dan Mašek, Apr 26 '18 at 22:51
@ZeyadEtman Does [this](https://pastebin.com/HcEeH1tn) do something close to what you want? -- I just find the dominant hues, create masks of the areas that contains pixels of that hue, and then use the masks to create individual images. It disregards the transitions between the areas that are of slightly different hue. You could use the masks to determine bounding boxes. — Dan Mašek, Apr 26 '18 at 23:21
it works! You made my day :) Please write it in answer to upvote it :) Thanks So much. @DanMašek — Zeyad Etman, Apr 26 '18 at 23:27
@ZeyadEtman Glad to help. I'll add some cropping of the detected "blobs" and will write up and answer. :) — Dan Mašek, Apr 26 '18 at 23:31

score 9 · Accepted Answer · answered May 01 '18 at 14:45

The approach is fairly straightforward. We begin by converting to HSV and grabbing only the hue channel.

image_hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
h,_,_ = cv2.split(image_hsv)

Next, we find the dominant hues -- first count the occurrences of each hue using numpy.bincount (we flatten the hue channel image to make it one-dimensional):

bins = np.bincount(h.flatten())

And then find which ones are common enough using numpy.where:

MIN_PIXEL_CNT_PCT = (1.0/20.0)
peaks = np.where(bins > (h.size * MIN_PIXEL_CNT_PCT))[0]

Now that we've identified all the dominant hues, we can repeatedly process the image to find the areas correspond to each of them:

for i, peak in enumerate(peaks):

We begin by creating a mask which selects all the pixels of this hue (cv2.inRange, and then extracting the corresponding parts from the input BGR image (cv2.bitwise_and.

mask = cv2.inRange(h, peak, peak)
blob = cv2.bitwise_and(image, image, mask=mask)

Next, we find the contours (cv2.findContours of all the continuous areas of this hue, so that we can process each of them individually

_, contours, _ = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

Now, for each of the identified continuous area

for j, contour in enumerate(contours):

We determine the bounding box (cv2.boundingRect, and create a mask corresponding to just this contour by filling the contour polygon with white (numpy.zeros_like and cv2.drawContours)

bbox = cv2.boundingRect(contour)
contour_mask = np.zeros_like(mask)
cv2.drawContours(contour_mask, contours, j, 255, -1)

Then we can extra just the ROI corresponding to the bounding box

region = blob.copy()[bbox[1]:bbox[1]+bbox[3],bbox[0]:bbox[0]+bbox[2]]
region_mask = contour_mask[bbox[1]:bbox[1]+bbox[3],bbox[0]:bbox[0]+bbox[2]]
region_masked = cv2.bitwise_and(region, region, mask=region_mask)

Or visualize (cv2.rectangle the bounding box:

result = cv2.bitwise_and(blob, blob, mask=contour_mask)
top_left, bottom_right = (bbox[0], bbox[1]), (bbox[0]+bbox[2], bbox[1]+bbox[3])
cv2.rectangle(result, top_left, bottom_right, (255, 255, 255), 2)

Or do any other processing you want.

Full Script

import cv2
import numpy as np

# Minimum percentage of pixels of same hue to consider dominant colour
MIN_PIXEL_CNT_PCT = (1.0/20.0)

image = cv2.imread('colourblobs.png')
if image is None:
    print("Failed to load iamge.")
    exit(-1)

image_hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
# We're only interested in the hue
h,_,_ = cv2.split(image_hsv)
# Let's count the number of occurrences of each hue
bins = np.bincount(h.flatten())
# And then find the dominant hues
peaks = np.where(bins > (h.size * MIN_PIXEL_CNT_PCT))[0]

# Now let's find the shape matching each dominant hue
for i, peak in enumerate(peaks):
    # First we create a mask selecting all the pixels of this hue
    mask = cv2.inRange(h, peak, peak)
    # And use it to extract the corresponding part of the original colour image
    blob = cv2.bitwise_and(image, image, mask=mask)

    _, contours, _ = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

    for j, contour in enumerate(contours):
        bbox = cv2.boundingRect(contour)
        # Create a mask for this contour
        contour_mask = np.zeros_like(mask)
        cv2.drawContours(contour_mask, contours, j, 255, -1)

        print "Found hue %d in region %s." % (peak, bbox)
        # Extract and save the area of the contour
        region = blob.copy()[bbox[1]:bbox[1]+bbox[3],bbox[0]:bbox[0]+bbox[2]]
        region_mask = contour_mask[bbox[1]:bbox[1]+bbox[3],bbox[0]:bbox[0]+bbox[2]]
        region_masked = cv2.bitwise_and(region, region, mask=region_mask)
        file_name_section = "colourblobs-%d-hue_%03d-region_%d-section.png" % (i, peak, j)
        cv2.imwrite(file_name_section, region_masked)
        print " * wrote '%s'" % file_name_section

        # Extract the pixels belonging to this contour
        result = cv2.bitwise_and(blob, blob, mask=contour_mask)
        # And draw a bounding box
        top_left, bottom_right = (bbox[0], bbox[1]), (bbox[0]+bbox[2], bbox[1]+bbox[3])
        cv2.rectangle(result, top_left, bottom_right, (255, 255, 255), 2)
        file_name_bbox = "colourblobs-%d-hue_%03d-region_%d-bbox.png" % (i, peak, j)
        cv2.imwrite(file_name_bbox, result)
        print " * wrote '%s'" % file_name_bbox

Console Output

Found hue 32 in region (186, 184, 189, 122).
 * wrote 'colourblobs-0-hue_032-region_0-section.png'
 * wrote 'colourblobs-0-hue_032-region_0-bbox.png'
Found hue 71 in region (300, 197, 1, 1).
 * wrote 'colourblobs-1-hue_071-region_0-section.png'
 * wrote 'colourblobs-1-hue_071-region_0-bbox.png'
Found hue 71 in region (301, 195, 1, 1).
 * wrote 'colourblobs-1-hue_071-region_1-section.png'
 * wrote 'colourblobs-1-hue_071-region_1-bbox.png'
Found hue 71 in region (319, 190, 1, 1).
 * wrote 'colourblobs-1-hue_071-region_2-section.png'
 * wrote 'colourblobs-1-hue_071-region_2-bbox.png'
Found hue 71 in region (323, 176, 52, 14).
 * wrote 'colourblobs-1-hue_071-region_3-section.png'
 * wrote 'colourblobs-1-hue_071-region_3-bbox.png'
Found hue 71 in region (45, 10, 330, 381).
 * wrote 'colourblobs-1-hue_071-region_4-section.png'
 * wrote 'colourblobs-1-hue_071-region_4-bbox.png'
Found hue 109 in region (0, 0, 375, 500).
 * wrote 'colourblobs-2-hue_109-region_0-section.png'
 * wrote 'colourblobs-2-hue_109-region_0-bbox.png'
Found hue 166 in region (1, 397, 252, 103).
 * wrote 'colourblobs-3-hue_166-region_0-section.png'
 * wrote 'colourblobs-3-hue_166-region_0-bbox.png'

Example Output Images

Yellow bounding box:

Yellow extracted region:

Biggest green bounding box (there are several other small disjoint areas as well):

...and the corresponding extracted region:

How can I used this method for grayscale or single channel images? — Gunjan naik, Nov 26 '18 at 08:13
I get "TypeError: lowerb is not a numpy array, neither a scalar" on line mask = cv2.inRange(h, peak, peak) — Cybernetic, Oct 25 '19 at 15:03

score 1 · Answer 2 · answered Apr 28 '18 at 11:31

First step is to understand what your algorithm is doing...specifically this function: ret,thresh = cv2.threshold(img,127,255,0)

the value 127 is the greyscale value between 0 and 255. Threshold function changes pixel values below 127 to 0 and above 127 to 255

With reference to your coloured image, the greyscale output for both the green blob and yellow blob is above 127, so both of those are changed to 255, and hence both are captured by the findContours() method

You can run imshow on thresh object to understand exactly what is going on.

Now when you replace 127 with 200, only the yellow blob has a greyscale value above 200, so only that blob is seen in the thresh Mat

To detect "all objects" at once, please experiment further with threshold method and study the thresh object using imshow

Bounding box on objects based on color python

2 Answers2

Full Script

Console Output

Example Output Images

Linked