Stack Overflow
Stack Overflow

Questions tagged [model-comparison]

A task in statistics and machine learning for comparing models that are fit to the same data with the aim of determining which best explains the data. Questions about how to use model comparision to evaluate models are likely more appropriate for CrossValidated (https://stats.stackexchange.com)!

Model comparison is a task in model selection involving the application of a measure (criterion) to the fits of a set of candidate models to data. The resulting measurements enable a quantitative comparison of appropriateness of the models in explaining the data.

See the Model Selection Wikipedia entry.

98 questions
1
vote
1 answer

AIC & BIC of PyMC mixture model

I am using PyMC to fit some data to a straight line. The data have outliers, so I adapted some code (third example at the link) written by Jake Vanderplas for his textbook. The method uses a vector variable qi to encode whether each individual data…
DathosPachy
  • 742
  • 1
  • 6
  • 17
1
vote
1 answer

Finding AIC and proportion of explained variance for multinomial data

I've got a dataset with a multinomial response variable and continuous predictor variables, and am trying to find out the AIC, F, p value, and proportion of explained variance for each of the predictor variables for each category of the response…
Anthony BH
  • 11
  • 2
1
vote
1 answer

How to get AIC or BIC for multivariate multiple regression, in R

I'm trying to compare two multivariate multiple regression models in R (see here) When I use AIC() or BIC(), R says it does not allow multiple responses. Is there a way to get a single AIC/BIC or r^2 for a multivariate multiple regression model (or…
RandomMonitor
  • 439
  • 1
  • 8
  • 16
1
vote
0 answers

Finding best curve fit with AIC

I have a quite sophisticated problem (for me). I have written a code that calculates interpolation for a given data set. After that it calculates the Akaike information criterium to check which interpolation suite best. Looks like that: …
wiedzminYo
  • 531
  • 4
  • 11
  • 24
1
vote
1 answer

Model selection with AIC

I have fitted the normal distributions to my claim amount data using the fitdistr in R. How do i fit Multivariate normal distribution (Two dimensional normal distribution) ? I want to choose which one fits my sample data the best using the AIC in R.…
harry
  • 11
  • 4
1
vote
0 answers

SAS selecting top logit models by AIC

I have a problem with SAS proc logistic. I was using the following procedures when I had OLS regression and everything worked OK: proc reg data = input_data outest = output_data; model y = x1-x25 / selection = cp aic stop = 10; run; quit; Here I…
user3812923
  • 39
  • 1
  • 4
1
vote
2 answers

The best model, according to both AIC and BIC, contains only a non significant term

Id like to know how is it possible that the best model (m6), based on AIC and BIC values, can have a non significant term whereas the second best model(m5) has a signficant term. I have the following competing models…
Agus camacho
  • 868
  • 2
  • 9
  • 24
1
vote
1 answer

Loop in R to select lowest AIC for a statistical model

I want to write a loop to get the lowest AIC value for the following model by changing the degrees of freedom, like df=2 in the definition of varknots1. I just use random data here since I don't know how to upload my data. I am trying to learn to…
user26221
  • 33
  • 5
1
vote
2 answers

Cross-validation for model comparison

I have a relative big data: more than 370,000 observations, categorical dependent variable with 250 levels，10 independent variables which including both numeric and categorical variables. I want to perform a 10-folds cross-validation for model…
Archer
  • 63
  • 6
1
vote
1 answer

AIC values in model comparison

I was comparing two models using the AIC. However, I realized that both AIC values are too small (-4752.66, and the other is close to that). I was wondering if that is normal or I did something wrong while calculating it.
Günal
  • 751
  • 1
  • 14
  • 29
0
votes
0 answers

Power analysis: Model comparison with LRT in R

Any suggestions on how to best implement a power analysis in R when having a binomial generalized linear mixed model (glmm) with 2 categorical predictors as fixed effects (2 levels and 8 factor levels) and testing hypotheses through model comparison…
Sahila
  • 35
  • 5
0
votes
1 answer

Using Vuong Test for SEM with MLM;MLR (robust estimator) / under non normality

I have two strict non-nested models that I would like to compare. However, the normal distribution assumption is violated, so the models were calculated with a robust estimator (MLR).
Marvin Schittko
  • 3
  • 1
0
votes
0 answers

AICcmodavg::aictab error: Functions do not support mixture of model classes

I am trying to compare a set of models with poisson, zero-inflated poisson, negative binomial and zero-inflated negative binomial with the AICcmodavg::aictab function. The models with poisson ang negative binomial distribution goes well, but when I…
kirchoffs
  • 3
  • 3
0
votes
0 answers

Is it physical to consider model with number of free parameter higher than data point while model comparison in Ultranest bayesian analysis

I am conducting a model comparison for a limited set of data using the Ultranest nested sampling bayesian analysis. I realized that for a few complex models, the number of data points is less than the free parameters in the model. Is it physical to…
subhrata dey
  • 41
  • 3
0
votes
1 answer

Model comparison for glmmTMB objects with beta family

We are performing a beta mixed-effects regression analysis using glmmTMB package, as shown below: mod = glmmTMB::glmmTMB(data = data, formula = rating ~ par1 + par2 + par3 + (1|subject)+(1|item), …
listopad
  • 1
  • 1
1 2 3
4
5 6 7