0

I did 5x2 cross validation experiments and, after that, I did the Tukey-HSD pairwise comparison comparing the 10 accuracy of 5 techniques pairwise as shown below.

Tukey-HSD Pairwise Statistical test

The graphic above was yielded after the following R commands:

data <- read.table("experimento-geral.txt", head=TRUE, sep=",", dec=".");
data$metodo <- factor(data$metodo);
summary(data$acuracia)
aov.data <- aov(data$acuracia ~ data$metodo, data=data)
anova(aov.data)
tky <- TukeyHSD(aov.data, "data$metodo");
png("pertubacao-metodo.png",width=320,height=480)
plot(tky)

What I want to know is why a pair of techniques in this graph is represented by just one confidence interval.

What I know is that one technique can be represented with one confidence interval. Why in the graphic one confidence interval is representing a couple of techniques? is this the subtraction of the two confidence intervals?

mad
  • 2,677
  • 8
  • 35
  • 78

1 Answers1

3

I'm not quite following what you did prior to the the ANOVA nor whether that renders the results of the Tukey pairwise comparisons moot, but...

Consider the x-axis label. This indicates that the scale is in terms of the differences of means of acuracia values for pairs of metodo levels. So, for each pair of metodo we have a single estimate of the true difference in the mean values of acuracia for the two method considered in each pair. That estimate is subject to uncertainty. This uncertainty is reflected in the confidence interval for that pair's mean difference. The widths of these confidence intervals have been adjusted following Tukey's HSD method to control the Type I error rate over the set (family) of comparisons.

The key thing to grasp is that the comparison of each pair is reduced to the difference of the means of the observations for each pair. That is a single estimate per pair of methods and hence a single confidence for that single estimate.

Gavin Simpson
  • 170,508
  • 25
  • 396
  • 453