0

I am a newbie in machine Learning, i am building a complaint categorizer and i want to provide a feedback model so that it can improve over time

import numpy
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.naive_bayes import MultinomialNB
value=[
'drought',
'robber',
]
targets=[
'water_department',
'police_department',
]
classifier = MultinomialNB()        
vectorizer = CountVectorizer()
counts = vectorizer.fit_transform(value)

classifier.partial_fit(counts[:1], targets[:1],classes=numpy.unique(targets))
for c,t in zip(counts[1:],targets[1:]):
    classifier.partial_fit(c, t.split())

value.append('dogs')                                   #new value to train
targets.append('animal_department')                    #new target
vectorize = CountVectorizer()
counts = vectorize.fit_transform(value)
print counts
print targets
print vectorize.vocabulary_
####problem lies here
classifier.partial_fit(counts["""dont know the index of new value"""], targets[-1])
####problem lies here

Even if i somehow find the index of newly inserted value, it is giving the error 

ValueError: Number of features 3 does not match previous data 2.

even thought i made it to insert one value at a time

Aditya Agarwal
  • 453
  • 4
  • 11
  • Do you want to classify single words like 'dogs', 'draught' into target labels (classes) like 'animal', 'water', 'police' etc.? In that case, why not use a dictionary? – Debasis Aug 02 '17 at 20:58
  • I need to classify sentences that contain these words. – Aditya Agarwal Aug 03 '17 at 04:07
  • okay... so in that case... do you have an exhaustive list of words with known labels? i'm not clear of what you're wanting to achieve with the feedback step? this isn't doing a real feedback (user driven)... is that something that you'd want in your application and hence want to simulate it programatically? in summary, you'd want incremental training, if i'm not mistaken? – Debasis Aug 03 '17 at 13:04
  • Yes, i want incremental training and i want to simulate it programatically. – Aditya Agarwal Aug 04 '17 at 05:13

1 Answers1

1

I will try to answer the question from a general point of view. There are two sources of problem in the Naive Bayes (NB) approach described here:

  1. Out-of-vocabulary (OOV) problem
  2. Incremental training of NB

OOV problem: The simplest way to tackle the OOV problem is to decompose every word into character 3 grams. How many such 3-grams are possible? Assuming lower-casing there are only 26 possible ways to fill each place and hence the total number of possible character 3-grams is 26^3=17576, which is significantly lower than the number of possible English words that you're likely to see in text.

Hence, generally speaking, while training NB, a good idea is to use probabilities of character n-grams (n=3,4,5). This will drastically reduce the OOV problem.

Incremental training: For incremental training, given a new sentence decompose it into terms (character n-grams). Update the count of of each term for its corresponding observed class label. For example, if count(t,c) denotes how many times was the term t observed in class c, simply update the count if you see t in class 0 (or class 1) during incremental training. Updating the counts will update the maximum likelihood probability estimates as well.

Debasis
  • 3,680
  • 1
  • 20
  • 23