4

I have bootstrapped model fits to some data from different three groups, using the 2.5 and 97.5 percentiles to generate a 95% confidence interval.

I know that if 95% confidence intervals do not overlap then there there is a significant difference between values of at least p<0.05. I would like to calculate the exact p values for pairwise comparisons between the following groups:

           Mean     LowerCI   UpperCI 
Group A:  0.05113   0.02279   0.07926

Group B:  0.10979   0.07954   0.14352

Group C:  0.10937   0.09400   0.13040

Any help would be greatly appreciated!

Siguza
  • 21,155
  • 6
  • 52
  • 89
X__
  • 89
  • 2
  • 8
  • How did you bootstrap the values? Did you compute the 95%CI to summarise the raw data, or to obtain the precision of some parameter following the bootstrap? In the former case, you could use the `aov` function to fit an analysis of variance model to the 3 groups, and then run the `TukeyHSD` function to get the pair-wise comparisons. – TWL Nov 21 '13 at 19:05

1 Answers1

0

NO! Don't do that!

Doing ANOVA on bootstrapped samples is a nonsense, in my opinion. I am still struggling to find a good way to solve this, but all what I've read tells me not to do that. Plus if you try it in practice you'll easily realize that doing the bootstrapping will give you an overly optimistic significance everywhere (ways too optimistic), due to the large sample of the bootstrapped.

Don't know the answer, but don't do ANOVA.

Lorenzo Menichetti
  • 11
  • 1