0

I fit a linear mixed-effects model using lmer from lme4 with the formula:

Response ~ Age + Gender + tOrder + Condition * Scale + (1|Participant)

I used estimate_contrasts from the modelbased package for R:

(mbc <- modelbased::estimate_contrasts(mdl.LMM,contrast="Condition",at="Scale"))

Level1 | Level2 | Scale | Difference |        95% CI |   SE |     df |     t |     p
L1     |     L2 |     A |       0.31 | [-0.02, 0.65] | 0.14 | 221.15 |  2.25 | 0.076
L1     |     L2 |     B |       0.19 | [-0.14, 0.53] | 0.14 | 221.75 |  1.41 | 0.320
L1     |     L2 |     C |       0.37 | [ 0.05, 0.70] | 0.14 | 221.57 |  2.75 | 0.013 
L3     |     L2 |     A |       0.18 | [-0.17, 0.53] | 0.14 | 221.39 |  1.27 | 0.414 
L3     |     L2 |     B |      -0.07 | [-0.43, 0.28] | 0.15 | 221.74 | -0.49 | 0.621 
L3     |     L2 |     C |       0.67 | [ 0.32, 1.01] | 0.14 | 221.56 |  4.66 | < .001
L3     |     L1 |     A |      -0.13 | [-0.47, 0.20] | 0.14 | 221.10 | -0.94 | 0.414 
L3     |     L1 |     B |      -0.27 | [-0.62, 0.08] | 0.15 | 221.75 | -1.84 | 0.200 
L3     |     L1 |     C |       0.29 | [-0.04, 0.63] | 0.14 | 221.50 |  2.10 | 0.037

The last row is what has me confused. I am unclear as to why the p value can be less than .05 while the 95% confidence interval includes 0. I would think that the 95% CI and the p-value should agree.

The result in the first row has a larger difference and t value, 95%CI closer to --- but still including --- 0, but has a p value > 0.05.

I note that estimate_contrasts uses a the Holm (1979) method to correct p-value for multiple comparisons by default.

What might explain this apparent discrepancy between the reported p-value and 95% CI?

Or if I am misunderstanding something, could someone kindly explain why this result is correct?

Thank you in advance.

user8189397
  • 11
  • 2
  • Seems weird, we could use a [mcve]. Perhaps a Gaussian or profile CI is being computed in one place and the p-value is based on a t-distribution? (Nevermind, with df = 221.5 that shouldn't matter) – Ben Bolker May 24 '23 at 20:22
  • Thanks for the edit, but this still isn't a reproducible example ... can you post your data set or use a built-in data set to come up with an example? – Ben Bolker May 24 '23 at 21:42
  • Unfortunately, I am unable to share the dataset. I imagine this is an uncommon result and therefore a built-in data set would not result in the same ambiguity. I appreciate your taking a look. – user8189397 May 25 '23 at 00:53

0 Answers0