3

How can I compare two CFA models estimated with DWLS / WLSMV?

To find out which CFA model fits best for my data, I used the DWLS estimator for ordinal data in lavaan and specified two models:

  • 4-factor-model, i.e. model.ordinalX
  • 2-factor-model, i.e. model.ordinalY

Code:

model.ordinalX = cfa(model.4, data=Data, ordered=c(
"AVf1","AVf2","AVf3","AVf4","AWf1","AWf2","AWf3","AWf4","ABf1","ABf2","ABf3","AAf4","AAf1","AAf2","AAf3","AAf4"))

summary(model.ordinalX, fit=T)

Subsequently, I specified the 2-factor-model. When comparing those two, lavaan returns an error.

Code:

anova(model.ordinalX, model.ordinalY)

The warning message is:

Error in lav_test_diff_af_h1(m1 = m1, m0 = m0) : lavaan ERROR: unconstrained parameter set is not the same in m0 and m1.

(Please note: If I use normal Maximum Likelihood I can compare the models.)

I've noticed that there is a difference in the outputs. With Maximum Likelihood estimation there are AIC and BIC values in the outputs, which are missing in the DWLS-outputs. These seem to be relevant for the comparison since the output of the comparison of the Maximum-Likelihood models contains those values.

L. Bakker
  • 147
  • 1
  • 13
  • I think we need to see the model code. These two models would not normally be nested. If you can make it reproducible, that will be even easier. – Jeremy Miles Jan 04 '17 at 18:44
  • It's easier if you add it to the question. What about the two factor? – Jeremy Miles Jan 06 '17 at 16:24
  • Models are nested if you can move from one model to the second model by adding additional constraints. CFA models are only nested if two factors become one factor by constraining factor correlations to 1 (and their covariances with other factors to be equal). But then perhaps you shouldn't treat them as nested. See Savalei, V., & Kolenikov, S. (2008). Constrained vs. unconstrained estimation in structural equation modeling. Psychological Methods, 13, 150-170. – Jeremy Miles Jan 06 '17 at 16:27
  • I think your question is more statistical - I would ask it on CrossValidated, and put in the code. – Jeremy Miles Jan 06 '17 at 16:27
  • I would guess that it is simply inappropriate to compare models in the way the `anova` function does so lavaan doesn't allow it. I don't know the answer, but it seems like DWLS (also called WLSMV, I think) is not maximum likelihood, so a traditional likelihood ratio test is not appropriate. This M-plus book I found suggests there is a way to do a test, so I'd take a look at the literature on models fit using this approach. – ndoogan Jan 09 '17 at 15:31
  • Some documentation of lavaan seems to point to potentially appropriate literature. See the section "Satorra-Bentler scaled test statistic and robust standard errors" https://www.jstatsoft.org/index.php/jss/article/view/v048i02/v48i02.pdf – ndoogan Jan 09 '17 at 15:48
  • I don't know. I just wanted to give you a lead to follow. The underlying research may give you some hints about how to compare the models, potentially using your own coded procedure that follows a guideline laid out by someone who knows what they are doing. Unfortunately (or is it fortunate?), we often can't count on the software to do everything for us. – ndoogan Jan 09 '17 at 16:05

0 Answers0