0

I have come up with a text recognition algorithm. This algorithm recognizes text in natural images. I am trying to test it against the groundtruth available for the dataset of ICDAR's robust reading challenge. For this, I have generated an xml file containing coordinates of text regions in a scene image, as recognized by my algorithm. A similar xml file is provided for the groundtruth data.

To generate quantitative results of comparison of the two xml files, i am required to use DetEval software (as mentioned in the site). I have installed a command line version on linux.

The problem is: DetEval is not reading the input xml files. Specifically,

I run the following command (As per the instructions on the DetEval website): rocplot /home/ekta/workspace/extract/result_ICDAR_2011/txt/GT2.xml { /home/ekta/workspace/extract/result_ICDAR_2011/txt/final.xml }

Here, GT2.xml is the groundtruth and final.xml is the file generated by my algorithm.

I get the following error message: evaldetection -p 0.8,0.4,0.8,0.4,0.4,0.8,0,1 "{" "/home/ekta/workspace/extract/result_ICDAR_2011/txt/GT2.xml" | readdeteval -p 1 - >> /tmp/evaldetectioncurves20130818-21541-1kum9m9-0 evaldetection -p 0.8,0.4,0.8,0.4,0.4,0.8,0,1 "{" "/home/ekta/workspace/extract/result_ICDAR_2011/txt/GT2.xml"I/O warning : failed to load external entity "{" Couldn't parse document { -:1: parser error : Document is empty ^ -:1: parser error : Start tag expected, '<' not found ^ I/O error : Invalid seek Couldn't parse document - rocplot: ERROR running the command: evaldetection -p 0.8,0.4,0.8,0.4,0.4,0.8,0,1 "{" "/home/ekta/workspace/extract/result_ICDAR_2011/txt/GT2.xml" | readdeteval -p 1 - >> /tmp/evaldetectioncurves20130818-21541-1kum9m9-0Error code: 256

What do i do? I am positive that there is no error in generating my xml file because even the groundtruth file obtained from the website is not being parsed. Please help!

Regards Ekta

Ekta
  • 507
  • 1
  • 5
  • 11

1 Answers1

2

So, I managed to solve this issue. Turns out I was giving the wrong commands. rocplots is to be used only when I need to have multiple runs on the ground truth and detection files with varying evaluation parameters. See this paper to know more about the parameters involved.

Currently, I have one ground truth file and one detection file and I need to run it using just the default parameters used by DetEval. So, here is what needs to be done:

Go to directory where you have detevalcmd directory and enter detevalcmd directory. Run the following commands in that directory:

  1. ./evaldetection /path/to/detection/results/DetectionFilename.xml /path/to/ground/truth/file/GroundTruthFilename.xml > /path/where/you/want/to/store/results/result.xml

This will store the results in result.xml. Next, run the following command: 2. ./readdeteval /path/where/you/stored/results/result.xml.

This will give something like: 

**100% of the images contain objects.
Generality: xxx
Inverse-Generality: xxx
  <evaluation noImages="xxx">
    <icdar2003 r="xxx" p="xxx" hmean="xxx" noGT="XXX" noD="xxx"/>
    <score r="Xxx" p="xxx" hmean="xxx" noGT="xxx" noD="xxx"/>
  </evaluation>**

So, there you go! you got the recall, precision etc. for you algorithm.

Ekta
  • 507
  • 1
  • 5
  • 11