0

We have built a text classification solution using Naive Bayes with decent prediction accuracy.

In cases , where prediction has failed , we are displaying the prediction probability and also we are manually pulling all matching text from the training data for the failed test case, which is displayed to the user to support the prediction logic .

This is a very good feature and helps in decision support.

The pulling of matching text from training data is currently done manually which will not scale for huge data sets .

I presume using Spark ML Lib , there is no way to pull matching text in order to support the prediction logic. Is there any other solution to pull up matching sentences ?

lives
  • 1,243
  • 5
  • 25
  • 61
  • Is your goal to find similar texts across the training set? Or is it more about getting text corresponding to a specific feature vector. – Alexey Svyatkovskiy Jan 22 '17 at 22:28
  • Goal is to find similar text across the training sets. This is to support and verify the prediction. We have manually implemented this solution by selecting the closest matching texts from the training set – lives Jan 25 '17 at 07:43

0 Answers0