2

I checked the accuracy of a segmentation method using the bboxPrecisionRecall function in Matlab version '9.4.0.857798 (R2018a) Update 2' and test result of an algorithm using IESK-ArDB dataset. The database is freely available here. Samples of database images here and here. I get 0s as output when trying to calculate the accuracy. What shall I do to get real results for my segmented algorithm?

The Code is below:

%% clean Workspace
clear;
clc;
%% my segmented bounding box cell
propied = {[48.5,84.5,102,59];[169.5,71.5,96,77];[251.5,114.5,47,51]}
%% Read Image
im = imread('t_A01_010.bmp');
imshow(im)
hold on
%% Ground truth standerd boxes.
%[GTruth,txt,raw] = xlsread('demo.xlsx');
groundTruthBoxes = [235 102 301 170;164 66 267 153 ;43 80 153 148]
%Convert bounding boxes from struct to cell.
boundingBoxes = propied;

% Convert cell to Matrix
bb = cell2mat(boundingBoxes(:));
% Move rows up down and fix matrix numbers
bb1 = fix(flipud(bb))
% Draw rectangle boxes for segmented Algorithm
for i=1:3
    rectangle('Position',bb1(i,:),'EdgeColor','y');
end
% Draw rectangle boxes for Standerd Ground Truth
for i=1:3
    rectangle('Position',groundTruthBoxes(i,:),'EdgeColor','g');
end    
%Evaluate the overlap accuracy against the ground truth data.
[precision,recall] = bboxPrecisionRecall(bb1,groundTruthBoxes)

segmented image Result on Command window

Adriaan
  • 17,741
  • 7
  • 42
  • 75
N.white
  • 67
  • 10

1 Answers1

2

It's because of the detection ratio treshold.

The third input to the function (default 0.5) specifies the minimum overlap between 2 boxes to consider them "matching". Your boxes are so different in size that the method assumes they are simply not matching, i.e. not looking at the same thing. You can change this value to vary the output.

For example:

[precision,recall] = bboxPrecisionRecall(bb1,groundTruthBoxes,0)
precision =

     1


recall =

     1

or

[precision,recall] = bboxPrecisionRecall(bb1,groundTruthBoxes,0.1)
precision =

    0.6667


recall =

    0.6667
Ander Biguri
  • 35,140
  • 11
  • 74
  • 120
  • @N.white there is no scientific definition of acceptable result, that's a human parameter. You are a human, so you should decide what is acceptable. – Ander Biguri Jul 10 '19 at 12:06
  • I mean for two outputs(0 or 0.1), the third input for the function bboxPrecisionRecall() . Is it sensitive to a large dataset accuracy or not sensitive? – N.white Jul 10 '19 at 14:19
  • @N.white yes, I know exactly what you meant. You are the only one that can answer that question. How much difference between 2 boxes is "too bad", how much is enough to consider them the same box? Only you can answer that. Your yellow boxes are **very** different to your green boxes. How big can a green box be for you to consider them "bad"? If they can be arbitrarily big, then make all of them as big as possible and will still be good enough. Is there an upper threshold that you find unacceptable? Those are questions you need to ask yourself – Ander Biguri Jul 10 '19 at 14:41
  • Thank you. I will take your questions on my mind as testing the results. only that green boxes are ground truth (standard) and yellow ones for the segmented algorithm. – N.white Jul 11 '19 at 12:51