Stack Overflow
Stack Overflow

Questions tagged [model-comparison]

A task in statistics and machine learning for comparing models that are fit to the same data with the aim of determining which best explains the data. Questions about how to use model comparision to evaluate models are likely more appropriate for CrossValidated (https://stats.stackexchange.com)!

Model comparison is a task in model selection involving the application of a measure (criterion) to the fits of a set of candidate models to data. The resulting measurements enable a quantitative comparison of appropriateness of the models in explaining the data.

See the Model Selection Wikipedia entry.

98 questions
0
votes
1 answer

Model comparison with RMSE

I am newby on data science and would like to ask for help of model selection. I have built 8 models to predict Salary vs year exp, position name and location. Then, I tried to compare 8 models by RMSE. But finally, I am not sure that which model I…
ธนาคาร ฉั่ว
  • 17
  • 1
  • 3
0
votes
2 answers

How to see the performance of all gam models when model select=TRUE

I run a gam with variable selection. But I want to evaluate the output of all the variables combination and not only for the best model for comparison. I am using mgcv package in R, is there some command for model evaluation (before I start coding…
Gabriela
  • 21
  • 5
0
votes
2 answers

MuMIn::dredge(). Exclude models only including main effect, keep only models with interactions

Is it possible to make a general logical statement for subsetting away (exclude) all potential models that only include the main effect of e.g. A? y ~ A + B + C + A:C + A:B For models including A, only include those where A is a part of an…
ego_
  • 1,409
  • 6
  • 21
  • 31
0
votes
2 answers

How to test significant improvement of LRM model

Using the rms package of Frank Harrell I constructed a predictive model using the lrm function. I want to compare if this model has a significant better predictive value on a binomial event in comparison with another (lrm-) model. I used different…
Klaas J
  • 41
  • 3
0
votes
1 answer

How can I return the percentage frequency of the best lag chosen in VAR(2) Simulation chosen by Akaike Criteria

If I put a counter loop before simulating a 3-dimensional VAR(2) model and after computing lets say AIC and lag 3 is chosen as the best fitted lag then I had difficulty in calculating the percentage frequency of the best lag (or in other words how…
JSS11
  • 1
0
votes
2 answers

R mtcars dataset model selection - model changes dramatically when including am

I`m trying to do some model fitting for the mtcars dataset. I did a model with only transmission included which gives me this : Since the adj R^2 is only .338 I was looking for another model. To do this, I sarted by fitting all possible…
user3443063
  • 1,455
  • 4
  • 23
  • 37
0
votes
0 answers

Akaike information criterion (AIC) stepwise regression - same with different variables

I've been running an AIC in order to select variables for stepwise regression models that I have been running. The data is landscape data were parts of the landscape in 4 radii (1km, 2km, 3km and 4km) around 19 test locations have been intersected…
Axel Rösvik
  • 13
  • 2
  • 7
0
votes
0 answers

How to find best GLModel based upon AIC score

I am trying to find the best GLModel (based just opon AIC score at the moment). The data I work with are accessible here: https://drive.google.com/open?id=0B5IgiR_svnKcY25TQ29ZMGN3NVE Here below is the piece of code that fails to return the best…
New2coding
  • 715
  • 11
  • 23
0
votes
1 answer

AIC for logistic/ordinal regression

Can you use Akaike Information Criterion (AIC) for model selection with either logistic or ordinal regression? I want to create multiple different logistic and ordinal models to find the best fitting model for my data, and want to know if (AIC) will…
Alex
  • 11
  • 2
0
votes
0 answers

automated drop1 in R according to AIC

First of all I am fairly new to R. So please do not be so harsh in your comments. Thanks in advance for any help/link/how-to/example; those will be much appreciated. I have a model like this: b <- lmer(metric1~a+b+c+d+e+f+g+h+i+j+k+l+(1|X) + (1|Y),…
borgs
  • 3
  • 5
0
votes
2 answers

How to print AIC or BIC from ARIMA Model

I've created an ARIMA model, but I am unable to find a way to print the AIC or BIC results. I need these numbers for model comparison. Unfortunately the documentation on sourceforge is down, and I cannot find my answer when looking at the statsmodel…
Evy555
  • 215
  • 2
  • 9
  • 19
0
votes
1 answer

How to find AIC values for both models using R software?

I'm studying survival analysis. I estimated both Cox regression model and Buckley&James regression model. In order to determine which model is better for my dataset, I used Akaike Information Criteria (AIC). Well, How to find AIC values for both…
1190
  • 365
  • 2
  • 10
0
votes
1 answer

Compare two models using Anova in SAS

Is there a way in SAS to compare two regression models using ANOVA. What i want to replicate is - in R if i have 2 models - model1 & model2 i can directly run anova(model1, model2) to find if there is a significant difference between the two. Is…
Raj
  • 1,049
  • 3
  • 16
  • 30
0
votes
2 answers

Poisson regression AIC table

I am carrying out a series of poisson regression in R and then ranking my ranking my models based on AIC. however I get this as a result: > aictab(cand.set = Cand.models, sort = TRUE) Model selection based on AICc : K AICc Delta_AICc…
Guest
  • 17
  • 7
0
votes
0 answers

R - Repeated model fitting with variable deletion

I have random sample containing 1 response variable and 10 explanatory variables (X) and I'm trying to find the best subset applying linear regression No problems with fitting the model, but I need y^s for each subset, so I'm open to any suggestions…
Ondra Lepša
  • 23
  • 3
1 2 3 4 5
6
7