-1

I am just starting off focusing on the research about image caption, which is a sub-domain of CV.As we all know, like other deep learning traing process, you've got to train the model based on training set and once the model is ready, you can use it. So I have a question in the image caption problem, what will happen if the trained model encouter some object that never occur in the dataset? Thanks for you replies!

Michael Hsu
  • 7
  • 3

1 Answers1

0

It's likely 1 out of 2 things will happen:

  1. The object is classified as something within the dataset categories and the text generator will pick up this class to build a sentence.

  2. The object is not recognized and the rest of the frame is used to generate a sentence.

It depends how closely related the actual class is to anything that's built into the dataset. The first option is still more likely if the object dominates the area within the frame. You could set a manual threshold that discards class information below a certain confidence.

pietz
  • 2,093
  • 1
  • 21
  • 23
  • Thanks for your answering.But for the second option, I don't get it very much.If the object cannot be recogined, how the rest of frame can generate a sentence based on the unknown object? – Michael Hsu Sep 03 '18 at 10:58
  • This highly depends on your problem. Since you didn't give any practical context, I cannot make up an example that may be relevant for you. In theory, the frame could include other objects or it might contain information about the scene, time of day or other references. Your original question is very broad and highly theoretical. The only precise answer someone could give you is: Anything could happen. – pietz Sep 03 '18 at 11:13
  • Ok, maybe when I dig deepr in this domain, I may find answer.But you've pointed me a direction, thanks for your work. – Michael Hsu Sep 03 '18 at 11:18