3

Suppose i have trained my classifier and i want to find the right sense of a word in a sentence. One feature people use is called collocation where you consider words to the left/right of the confusing word and position is important . I am curious why this approach works? What information does considering collocations give us that helps us in text classification? Moreover, why is the position important

Programmer
  • 6,565
  • 25
  • 78
  • 125
  • @rrenaud: The homework was to use collocations to do text classification. I have done that . However, making an attempt to actually understand things, i want to know why sth worked given that it has worked. Just to let you know, you can do the above by building a LM consisting of n grams of size 2. Now P(w1w2|word) = Count(wordw1w2)/Count(word) assuming forward collocation – Programmer Nov 12 '11 at 11:19
  • @everyone: no answers??Please vote up if confused so that everyone can see – Programmer Nov 13 '11 at 17:14

1 Answers1

0

Check some details on why word sense disambiguation using collocation works here: http://en.wikipedia.org/wiki/Yarowsky_algorithm It's basically a Bayesian network.

Bogdan
  • 934
  • 7
  • 13