Stack Overflow
Stack Overflow

Questions tagged [model-comparison]

A task in statistics and machine learning for comparing models that are fit to the same data with the aim of determining which best explains the data. Questions about how to use model comparision to evaluate models are likely more appropriate for CrossValidated (https://stats.stackexchange.com)!

Model comparison is a task in model selection involving the application of a measure (criterion) to the fits of a set of candidate models to data. The resulting measurements enable a quantitative comparison of appropriateness of the models in explaining the data.

See the Model Selection Wikipedia entry.

98 questions
1
vote
0 answers

ANOVA result is inconsistent (AIC VS deviance)

I am working on GLM models (using glmer). Now I am exploring whether I need an interaction term. I'd like to find the best model, but the following result is confusing: Models: g1: y ~ year + (1 | BZR/PLR) g2: y ~ year + year * BZR + (1 |…
Aekyung
  • 11
  • 3
1
vote
1 answer

Does using the same trainControl object for cross-validation when training multiple models with caret allow for accurate model comparison?

I have been delving into the R package caret recently, and have a question about reproducibility and comparison of models during training that I haven't quite been able to pin down. My intention is that each train call, and thus each resulting…
wcanners
  • 35
  • 7
1
vote
0 answers

Model selection using glmulti

I am attempting to run glmulti to test all possible subsets for model selection. The following is the code that I am trying to use. lmer.glmulti<-function(formula, data, random="", ...){ lmer(paste(deparse(formula),random),data=data, …
Kelsey Spencer
  • 57
  • 1
  • 1
  • 9
1
vote
1 answer

Why isn't model tuning done before model selection?

I have observed in many articles and books that model selection is done before model tuning. Model selection is generally done using some form of cross validation like k-fold where multiple models' metrics are calculated and the best one is…
Mohit Shah
  • 843
  • 1
  • 6
  • 20
1
vote
1 answer

model selection with unrestricted model in midasr package

I have a monthly time series and two weekly time series and I want to use the MIDAS regression using the midasr package in R. Furthermore, I am using the unrestricted model, where six lags of the monthly variable and one lag of every weekly variable…
user155417
  • 55
  • 1
  • 7
1
vote
1 answer

Writing log likelihood for WAIC (logistic hierarchal stan model)

I'm creating a new model and I want to compare this with another model using WAIC. I understand that I need to write a generated quantities block. However, I'm struggling to convert the logsumexp of beta. I would greatly appreciate any leads/help.…
Ruchi Ookalkar
  • 15
  • 2
1
vote
0 answers

Comparing AIC for different types of models (beta and normal)

I have responses which are proportions mainly centered around 0.6-0.7, and not many of them are close to 0 or 1. I have tried fitting both normal and beta models, and the normal models yield lower AIC than the beta models. I use the lm package for…
SummerRed
  • 11
  • 1
1
vote
0 answers

Comparison of mixture distribution models (computed in mixdist; R) using AIC

I am using the mixdist package to model length-frequency data into separate cohorts very similar to the method posted here (specifically the function mix()). Currently I have been looking at the size frequency histograms of my original data,…
jessjesseee
  • 11
  • 3
1
vote
1 answer

Optimizing word2vec model comparisons

I have a word2vec model for every user, so I understand what two words look like on different models. Is there a more optimized way to compare the trained models than this? userAvec = Word2Vec.load(userAvec.w2v) userBvec =…
Thingamajig
  • 4,107
  • 7
  • 33
  • 61
1
vote
0 answers

glmmPQL model comparison and selection in R

I am working with menstrual cycle data, and I want to investigate if carrying an infection predicts the occurence of pre-menstrual symptoms. In addition, I have no a priori reason to think that the pre-menstrual phase lasts for 3, 4 or 5 days (or…
Alex
  • 31
  • 2
1
vote
1 answer

model selection - mclust

I conducted latent class/cluster analysis in R using the package MCLUST. I have a revise and resubmit for my paper, and the reviewer suggested making a table of the fit indices for the cluster solutions (as of now I just reported BIC in the text).…
J Cole
  • 11
  • 2
1
vote
1 answer

AIC on nls on R

I have a problem to calculate the AIC. Indeed, I estimate the parameters of my 3 models: "mod_linear", which is a linear model and "mod_exp" and "mod_logis" which are two non linear models. I used the function…
Marambo
  • 57
  • 2
  • 7
1
vote
1 answer

Model comparison in PyMC3

I am new to PyMC3 and am trying to implement the hierarchical model from Kruschke (2015) section 12.2.2 (model comparison). I succeeded in defining the full model and then looking at the differences of posterior parameter values (determine whether…
Jordi
  • 193
  • 9
1
vote
3 answers

Plotting model comparison statistics in R

I combined several data-frames into a data-frame dfc with a fifth column called model specifying which model was used for imputation. I want to plot the distributions by grouping them by model. dfc looks something like: (1000 rows, 5 columns) X1 …
user4805479
1
vote
1 answer

Determine best Arima model based on AIC

I'm trying to fit Arima model and see which order is the best based on AIC i Have the following for statement, my question is how to show the order of the model because it just gives me AIC values and can't determine which model,, mid.ts is time…
mms
  • 365
  • 1
  • 3
  • 12
1 2
3
4 5 6 7