6

If you have a bayes classifier trained for a set of classes, how to detect if the output is significant enough to choose a class? It would be useful for detecting samples wich can't be asigned to a class. I have tried testing if the class probability is above mean+2*stddev of the probabilities of all the clases, but I don't think it will be robust.

piotr
  • 5,657
  • 1
  • 35
  • 60

1 Answers1

3

You could consider log-likelihood ratios. Consider R(C) = log(P(C|D)/P(~C| D), where C is the class, D are the features. Then you probably want to ensure that R(C) is greater than some positive amount.

Il-Bhima
  • 10,744
  • 1
  • 47
  • 51
  • I'll look into this and do some tests – piotr Mar 10 '10 at 15:34
  • 1
    The likelihood ratio isn't going to help him here--at least not as you describe. He knows the *relative* "probabilities" P(C1|D), P(C2|D), ... , P(CN|D), but doesn't know how to normalize them properly because he has a non-exhaustive set of classes; i.e., SUM over i=1 to N of P(Ci|D) is NOT equal to unity because other unknown classes exist that contribute to the probability sum in unknown ways. Therefore, even though he can do a P(C1|D)/P(C2|D) likelihood ratio (the unknown normalization factor drops out), he CANNOT calculate P(~C|D) because his P(Ci|D) values aren't true probabilities. –  Jan 04 '12 at 07:13