0

I want to get Bayes factors for ANOVAs that are analogous to the classical F-tests, and I just want to make sure I understand correctly how to write the syntax, especially regarding subject IDs.

For example, I have between-subject independent variables a_between and b_between, and within-subject variables c_within and d_within, dependent variable values, with subject_id to identify each subject; in a dataset my_data.

If I understand correctly, for a full ANOVA, I should use:

anovaBF(values~a_between*b_between*c_within*d_within+subject_id, data = my_data, whichModels="bottom", whichRandom="subject_id") # and I assume the order of variables does not matter, e.g. it could also be d_within*a_between*c_within*b_between+subject_id

For only within-subjects ANOVA, I should use:

anovaBF(values~c_within*d_within+subject_id, data = my_data, whichModels="bottom", whichRandom="subject_id")

For only between-subjects ANOVA, I should use:

anovaBF(values~a_between*b_between, data = my_data, whichModels="bottom", whichRandom="subject_id")

So in the last case I have no +subject_id - otherwise I get Error in base::try(expression, silent = silent) : not enough observations. (Perhaps because there is only a single line per subject_id?)

Two main questions:

  1. Regardless of reasons, are the solutions above corrects?
  2. If the solutions are correct, why do I have to specify subject ID twice (once as whichRandom and once in the beginning as +subject_id) for within-subject variables, and why not when there are only between-subject variables?

(FYI, there is a related question with answer, but not exactly what I want to know: https://stats.stackexchange.com/questions/230224/mixed-bayesian-anova-using-bayesfactor-package-in-r)

gaspar
  • 898
  • 1
  • 13
  • 26

1 Answers1

1

From https://forum.cogsci.nl/index.php?p=/discussion/5203/bayesfactor-anovabf-syntax:

  1. Generally, yes - but I'm not sure you want to use whichModels = "bottom" - it is advised to stick with the defaults here (whichModels = "withmain"). Also you can't really get a BF for an F test - as BF are always comparative, so if you want a BF for each "effect" you'll need to think which comparison of which two models might represent that (like in step-wise hierarchical regression). Or, you may want to try to compute Inclusion BFs via bayestestR::bayesfactor_inclusion()(equivalent to JASP's effects panel).

  2. anovaBF isn't really an anova at all - it is actually a linear mixed model. So you need to specify +subject_id as it is an effect in your model, but you also need to tell anovaBF that it is a random effect (and not a fixed one).

Further helpful links:

https://forum.cogsci.nl/index.php?p=/discussion/2426/type-of-sums-of-squares

https://www.cogsci.nl/blog/interpreting-bayesian-repeated-measures-in-jasp

In any case, I'll stick to using bayestestR::bayesfactor_inclusion() with match_models = TRUE; that seems the most straightforward to me.

gaspar
  • 898
  • 1
  • 13
  • 26