2

I am new to the field of computer vision, so I apologise if the question is inappropriate in any way.

I have created a segmentation model using the PascalVOC 2012 data set, and so far I have only been able to test it on the train and val data sets. Now I wanted to test my model using the test set, however, it does not provide any annotations, so I am unsure of what I could do in order to measure my model's performance on the test data.

I have noticed that other data sets, such as COCO, do not provide the annotations for the test data.

I am curious how researchers who have published papers regarding models trained on these data sets have tested them on the test data in such case and what I can do in order to do the same.

Thanks in advance for your help!

MrFlairDigest
  • 43
  • 3

1 Answers1

3

The main reason why many of the major datasets do not release the test set is to avoid people reporting unreliable results due to overfitting.

For model selection and "informal" evaluation, you should split your training set into train and validation split, and evaluate on the latter while training only on the first.

So how is that researchers report results on the test set in the papers?

Once you have a definitive model you want to evaluate, you can upload your results to an evaluation server; in this way you can benchmark yourself w.r.t. the state-of-the-art without having explicit access to the test set.

Examples:

As a side note: VOC2012 is pretty old, so maybe you can also find the test set if you really need it. Take a look at this mirror by Joseph Redmon for example.

ndrplz
  • 1,584
  • 12
  • 16