I'm trying to evaluate the performance of a model using Pycocotools, it requires that you prepare the ground truth (gt) bboxes and detections (dt) bboxes as json files in a COCO style format. I worked a lot on creating those json files and avoided any missing bboxes, overlaps, and made sure that all the bboxes in the ground truth and detections are matched to the best of my knowledge as shown below: This is a sample of the json file for both the gt and dt bounding boxes that I am working on:
ground truth images key sample: enter image description here
ground truth annotation key sample: enter image description here
Detections images key sample: enter image description here
Detections annotations key sample: enter image description here
1- My first questions is: Is there anything wrong with the format of these json files?
After using the Pycocotools to get the evaluation metrics like the average recall and precision using the following code:
```
import os, json
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from pycocotools.coco import COCO
from pycocotools.cocoeval import COCOeval
input_dir = "./"
with open(os.path.join(input_dir, 'small pred.json'), 'r') as file:
json_str = file.read()
data = json.loads(json_str)
anns = data["annotations"]
print(type(anns))
print(anns[:5]) # Print the first 5 annotations for debugging purposes
coco_gt = COCO(os.path.join(input_dir, 'small labels.json'))
coco_dt = coco_gt.loadRes(anns)
imgIds = coco_gt.getImgIds()
cocoEval = COCOeval(coco_gt,coco_dt,'bbox')
cocoEval.params.imgIds = imgIds
cocoEval.evaluate()
cocoEval.accumulate()
cocoEval.summarize()`
```
I got bad results of the evaluation even after making the IOU threshold between (0.25-0.5) as follows: enter image description here
I tried to use the ground truth with the ground truth to make sure that the evaluation code that I am using is working, and I got good results but not perfect even when I am using the same bounding boxes on both sides My main question is why I am getting 0s for both the average precision and average recall evaluation values as shown above, even though I made sure that everything is matched between the gt bbox and the dt bboxes, and even after testing on 10 images only?
Here is an example of the images that I working on where I am detecting pothols (ground truth in blue and detections is red) enter image description here
I tried to use the ground truth with the ground truth to make sure that the evaluation code that I am using is working, and I got good results but not perfect even when I am using the same bounding boxes on both sides. I tried with different ranges of IOU, and rebuilt the json files to make sure that everything is matched and that I got the same number of images in both files, and tested on 10 images only, and tested on other 10 images but got the same results.
At the beginning there were some unmatched bboxes and I used to get -1 for recall and precision which means false positive based on my understanding, and after cleaning all of that I got 0s for all values which means there are no detections based on my understanding as well.
