How can I search a dictionary for a nltk stem?

Question

I'm having an issue checking to see if a stemmed word exists in a dictionary. This is for some sentiment analysis work that I am doing. All I am getting back is this error here:

Traceback (most recent call last):
File "sentiment.py", line 369, in <module>
score += int(senti_word_dict.get(get_stem(word)))
TypeError: int() argument must be a string or a number, not 'NoneType'

Here is my code for the method to look for a stemmed word through NLTK:

def get_stem(word):
    st = SnowballStemmer("english")
    stemmed_word = st.stem(word)
    return '' if stemmed_word is None else stemmed_word

Here is the code for checking for that word against the dictionary:

for comment in all_comments:
    score = 0
    tokens = tokenize(comment)
    for word in tokens:
      if word in senti_word_dict:
        score += int(senti_word_dict.get(get_stem(word)))
    print(str(score)+" "+comment)
    print('\n')

For now I am just getting the score. Is there a way that I can pass that stemmed word as a string to see what the score is in the dictionary? If there is anything I am doing wrong or could do better let me know! Thanks!

`stem_word` is never `None`, because that's the name of your function. You must use `stemmed_word`. Lesson: do not use similar identifiers. — DYZ, Mar 05 '17 at 06:01
Please update your code, because you essentially get a different question now. — DYZ, Mar 05 '17 at 06:04
As mentioned before. I am still getting the same error. I will update the post though. — Jacob Reed, Mar 05 '17 at 06:05

score 0 · Accepted Answer · answered Mar 05 '17 at 06:09

0

You check if word is in senti_word_dict. Perhaps it is. But then you stem it (it becomes a different word!) and attempt to retrieve the stem from the dictionary with senti_word_dict.get. If the stem is not in the dictionary (why should it be?), get() returns a None. Thus, the error. Solution: first stem the word and only then look it up.

answered Mar 05 '17 at 06:09

DYZ

55,249
10
64
93

Ah beautiful! Works now. Guess I needed a second set of eyes to see my dumb mistake. Thanks for the input! Late night coding probably isn't best for me haha. – Jacob Reed Mar 05 '17 at 06:13

How can I search a dictionary for a nltk stem?

1 Answers1