0

I want to calculate the area of predicted masks from the output of Detectron2 object detection Segmentation So when I run inference it returns the dictionary outputs = predictor(im)

pred_mask,pred_boxes,pred_scores. When I print the pred_masks the values are in binary (False,True)

But when i call the visualizer function by sending the predictions it paste all the predicted masks but it shows me (true false ) I don't get it Here is a coed of visualizer

v = Visualizer(im\[:, :, ::-1\],
metadata=grain_metadata,
scale=0.5,
instance_mode=ColorMode.IMAGE_BW
)
out = v.draw_instance_predictions(outputs\["instances"\].to("cpu"))
cv2_imshow(out.get_image()[:, :, ::-1])

This function is in a file

(detectron2->utils->visualizer->draw_instance_predictions(predictions)

I made some changes in that file in collab like trying to print the masks but it didn't affect at all I comment on the whole file but still the visualizer working can someone tell me how to get the masks values so I will draw on my own using OpenCV.

enter image description here

toyota Supra
  • 3,181
  • 4
  • 15
  • 19
Talha Gaming
  • 11
  • 1
  • 2
  • Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking. – Community Aug 03 '22 at 12:43
  • What exactly is the problem? `pred_masks` values are binary because it is a [binary mask](https://homepages.inf.ed.ac.uk/rbf/HIPR2/mask.htm#:~:text=A%20mask%20is%20a%20binary,zero%20in%20the%20output%20image.). So, the values you see printed are correct. – zepman Aug 09 '22 at 08:48
  • yes i want to get the polygons and draw by my self ? i dont know how to do this – Talha Gaming Aug 09 '22 at 09:49

1 Answers1

3

The values in pred_masks values are binary because it contains binary masks.

Referring to the Detectron2 model output format, you can get the contours of every predicted mask using OpenCV's findContours.

import cv2
import numpy as np

# "outputs" is the inference output in the format described here - https://detectron2.readthedocs.io/tutorials/models.html#model-output-format

# Extract the contour of each predicted mask and save it in a list
contours = []
for pred_mask in outputs[0]['instances'].pred_masks:
    # pred_mask is of type torch.Tensor, and the values are boolean (True, False)
    # Convert it to a 8-bit numpy array, which can then be used to find contours
    mask = pred_mask.numpy().astype('uint8')
    contour, _ = cv2.findContours(mask, cv2.RETR_LIST, cv2.CHAIN_APPROX_NONE)

    contours.append(contour[0]) # contour is a tuple (OpenCV 4.5.2), so take the first element which is the array of contour points

contours can then be used to draw the objects using OpenCV's drawContours like below.

# "image" is the original BGR image

image_with_overlaid_predictions = image.copy()

for contour in contours:
    cv2.drawContours(image_with_overlaid_predictions, [contour], -1, (0,255,0), 1)
zepman
  • 671
  • 6
  • 14