How can I calculate Intersection Over Union in Detectron2?

Question

I am using Detectron2 for object detection. I have registered pascalvoc dataset and trained a model for detection. How can I calculate Mean IOU of my test dataset ? I know that detection2 has a predefined function for calculating IOU i.e. detectron2.structures.pairwise_iou

I have the ground truth bounding boxes for test images in a csv file. The csv file contains (filename,width,height,class,xmin,ymin,xmax,ymax). How can I parse both the bounding boxes in IOU function and display it in google colab.

This is my code where I am generating a prediction bounding box

from detectron2.utils.visualizer import ColorMode
import random

dataset_dicts = DatasetCatalog.get('/content/test')
for d in random.sample(dataset_dicts, 5):    
    im = cv2.imread(d["file_name"])
    outputs = predictor(im)
    v = Visualizer(im[:, :, ::-1], metadata=microcontroller_metadata, scale=0.8)
    v = v.draw_instance_predictions(outputs["instances"].to("cpu"))
    plt.figure(figsize = (14, 10))
    plt.imshow(cv2.cvtColor(v.get_image()[:, :, ::-1], cv2.COLOR_BGR2RGB))
    plt.show()

score 2 · Answer 1 · answered Oct 17 '20 at 12:12

2

You have access to the bounding boxes and the classes via

outputs["instances"].pred_boxes.tensor.cpu().numpy()
outputs["instances"].pred_classes.cpu().numpy()

Now what you have to do:

Do your inferencing on the picture
Load the ground truth for the picture from your csv file
Compare the classes and bounding boxes of your inference with your ground trouth (= IoU)

answered Oct 17 '20 at 12:12

Alexander Riedel

1,329
1
7
14

Can you please tell me how to parse both the numpy arrays ie. the ground truth and the predictor in detectron2.structures.pairwise_iou(boxes1: detectron2.structures.boxes.Boxes, boxes2: detectron2.structures.boxes.Boxes) → torch.Tensor – AMRUTA BADGUJAR Oct 20 '20 at 05:47
I dont really get what you mean but: how are the ground truth data stored in your csv? You have to pair the right picture in your csv with the picture you're inferecing on, and then pair the right categorial bbox to the ground truth bboxes and categories – Alexander Riedel Oct 20 '20 at 08:51

score 2 · Accepted Answer · answered Nov 15 '20 at 10:25

You need to do what Alexander said. So let's say that for a given image you have the ground truth boxes. These you can be represented as an N,4 numpy array. Let's call it bboxes_gt. and let's say your prediction gives you M bounding boxes.

Now you can convert bboxes_gt into a Box oobject and then use structures.pairwise_iou() to compute all IOUs in a pariwise fashion. This will give you an N,M matrix with all these IOUs. It would look something like this:

bboxes_gt = structures.Boxes(torch.Tensor(bboxes_gt))
bboxes_pred = outputs["instances"].pred_boxes
IOUs = structures.pairwise_iou(bboxes_gt, bboxes_pred)

How can I calculate Intersection Over Union in Detectron2?

2 Answers2

Linked