Stack Overflow
Stack Overflow

Questions tagged [model-comparison]

A task in statistics and machine learning for comparing models that are fit to the same data with the aim of determining which best explains the data. Questions about how to use model comparision to evaluate models are likely more appropriate for CrossValidated (https://stats.stackexchange.com)!

Model comparison is a task in model selection involving the application of a measure (criterion) to the fits of a set of candidate models to data. The resulting measurements enable a quantitative comparison of appropriateness of the models in explaining the data.

See the Model Selection Wikipedia entry.

98 questions
0
votes
1 answer

Subsetting in Dredge (MuMln)- avoiding a three way interaction

I have a model with several main effects and several interactions. I want to avoid any models that would only include the 3 interaction terms. So basically all variations of main effects and main effects with various interactions but not anything…
Rebecca
  • 1
  • 1
0
votes
0 answers

(Wald)Test for comparing two nested feols/plm models

I want to test if model_1 <- feols(auth ~ dummy_past1 + dummy_past2 + dummy_past3 | region + date_f, # Fixed Effects data=df) is just as good as model_2 <- feols(auth ~ i(dummy_past1,specific_regions) + i(dummy_past2,specific_regions) +…
Thomasel
  • 1
0
votes
0 answers

Model comparison from the results from the mixed() function of the afex package

I am new to R and lmer. We have a project examining the effects of mimicking others’ voice (mimicry vs nonmimicry, a between-subjects fixed effect) , modality (reading vs listening, a within-subjects effect), valence (positive vs negative; a…
Cai Rendong
  • 1
0
votes
0 answers

loader constraint violation when launching an EMF compare comparison(and proper tools for model comparison.)

I'd like some insight on a couple of topics :"EMF compare" and "model to model comparison" in general. I am trying to achieve a comparison between two XMI models, which are different from each other (i) the first is an output model which is the…
narine narine
  • 21
  • 3
0
votes
1 answer

Is it appropriate to compare gamm (mgcv) models to get an interaction effect and how to do it?

I want to know if the development of volume over time (and I mean age, not waves) is different between groups. I also have some covariates. I made a simulation dataset: library(simstudy) def <- defData(varname = "volume", dist = "normal", formula =…
Keatars
  • 1
0
votes
0 answers

Same Testing Accuracy for Pruned model and Original model

I am trying to perform model pruning on ResNet50. However, I am getting exactly the same testing accuracy on both of them. What am I doing wrong? import tensorflow_model_optimization as tfmot import numpy as np prune_low_magnitude =…
Omkar Vedak
  • 25
  • 1
  • 5
0
votes
0 answers

Using Akaike information criterion (AIC) and Bayesian information criterion (BIC) for filtering PCA factors in mlr3?

I would like to use a PCA with subsequent feature selection/filtering in mlr3. I could not yet find AIC and BIC for this "filtering" in the package/framework. Is this because they do not fit conceptually, e.g. are all methods in mlr3filters…
ds_col
  • 129
  • 10
0
votes
0 answers

Comparison of mixed-effects model and a combination of two mixed-effects models

I hope this question fits into here because it is not only about a coding problem, but also digs into some theoretical questions concerning linear mixed-effects models. Assume the linear mixed-effects model: model1 <- lmer(RT ~ word_duration +…
hyhno01
  • 177
  • 8
0
votes
0 answers

extract(model, what = dic) from JAGS model returns NA for penalty

Using JAGS, I am fitting different models to data and would like to compare their fits using the deviance information criterion (DIC). I am using "run.jags" to fit a model and then "extract" to determine the DIC for the model after it runs. My…
H. Meredith
  • 13
  • 3
0
votes
0 answers

model comparison interpretation when chi square equals 0

I'm comparing two multilevel models in R using the anova() function. One model contains a control variable and another with an experimental variable. When I compare the two, I get a weird result where the chisquare is 0 and the p value is 1. I would…
J.Sabree
  • 2,280
  • 19
  • 48
0
votes
1 answer

mice's pool.compare gives "Error: No glance method for objects of class call" for lmerTest models

I'm trying to compare two models built using multiple imputations. When I try to compare the models, mice's pool.compare() gives the error that Error: No glance method for objects of class call or Error: unequal number of imputations for 'fit1' and…
J.Sabree
  • 2,280
  • 19
  • 48
0
votes
0 answers

What kind of Bonferroni adjustment to apply

I am currently working on sign predictability of certain non-linear models for various financial assets. The following table shows the percentage of correct sign forecasts for 5 financial asset generated using 9 different models where *, ** and…
Max
  • 21
  • 7
0
votes
1 answer

Decreasing AIC while incresing standard errors

I've got a problem chossing the right model. I have a model with various variables (covariables and dummyvariables). I was trying to find the best seize of the model, so I first started in comapring the models with AIC. From this it followed, that…
user734124
  • 489
  • 8
  • 20
0
votes
1 answer

Error message in model comparison using anova() in R

I am trying to compare the two models below H1 <- lm(y ~ x1 + x2, data = df) H2 <- lm(y ~ x1 + x2 + x3, data = df) anova(H1, H2) However, I get an error message: Error: Argument 'data' must be a data frame And when I define the data, then I…
user240313
  • 17
  • 6
0
votes
1 answer

Highlight the prediction line of the best fitting model in R?

I have fitted a few models to the same data. And have plotted the predction lines for all models in a single plot. Now I'd like to highlight (in bold, thicker line) the prediction line of the model with the lowest AIC. Can't seem to find resources…
Chiel
  • 47
  • 1
  • 8
1 2 3 4
5
6 7