1

In the PST package one can estimate the prediction quality of individual sequences using the log-loss, e.g:

R> ex2 <- c("a-a-b", "a-b-a-a-b", "b-b-b-b-a") 
R> ex2 <- seqdef(ex2) 
R> predict(S1.p1, ex2, output = "logloss")

logloss
[1] 0.9183 
[2] 0.7311 
[3] 0.9600

How do I compare these log-loss values statistically? Is there a way to show that 0.9183 is significantly different from 0.9600?

histelheim
  • 4,938
  • 6
  • 33
  • 63
  • This question would be better suited over [Cross Validated](http://stats.stackexchange.com/) where you may already find an answer [here](http://stats.stackexchange.com/questions/123401/confidence-intervals-for-the-log-loss-metric-for-model-comparison?answertab=active#tab-top) – Gilbert Mar 01 '17 at 17:03

0 Answers0