0

There is a fundamental question that I have not been able to fully understand. I have a 5 independent groups and want to know which of these group means are different. So I have two choices: 1. Do one-way anova and then post hoc test (eg. Tukey) 2. Just do multiple t test and then adjust p values , using pairwise.t.test Which approach should I go? what is the different between these two approach?

A followup question: If these 5 groups are dependent groups (repeated measure), then what is the correct way to do those comparison? Is there any difference from when those groups are independent?

Thank you in advance for the expert opinion.

zesla
  • 11,155
  • 16
  • 82
  • 147

1 Answers1

1

It mainly depends on whether your predictions about which groups are different from each other are a priori (made before you see the data) or are post hoc (only made after you see the data). If before you started collecting the data, you had specific predictions that some groups would be higher or lower than others, then you really should use preplanned contrasts (which can reduce to simple t-tests under some conditions).

If, on the other hand, you just thought that some of the groups might be different, but you don't know which ones, then you should do the overall ANOVA, and conduct some form of corrected test afterwards. Tests like Tukey's test and t-tests with a corrected p-value (Bonferroni) are both the same in the sense that they correct the level needed to consider a difference statistically significant. The difference is in the method used to make the correction.

Bonferroni corrects for multiple tests. Tukey (et. al) corrects for multiple tests and the post hoc nature of the tests.

You'll find this page useful for the question of how to do all this with repeated measures: David Howell's Page on How to do Multiple Comparisons with Repeated Measures

Steve Scher
  • 144
  • 1
  • 12