0

I have run a glm() model; but now I would like to measure the model's accuracy with PPV, NPV, sensitivity and specificity. However, I keep getting confusing results.

My outcome is a factor variable that looks like this: 

table(mydata$outcome)

   0    1 
6824  359

The predictors are a combination of continuous variables with 1 categorical (gender).

My code is: 

# To run the logistic model 

    mod <- glm(outcome~predictor1+predictor2+predictor3,data=mydata,family=binomial("logit"))
    summary(mod)

# To run predict() to get the predicted values of the outcome 

    predicted = predict(object =  mod, newdata=mydata, type = "response")

The results for this look like this: 

head(predicted)
         1          2          3          4          5          6 
0.02568802 0.02979873 0.01920584 0.01077031 0.01279325 0.09725329

This is very surprising as I was expected to observe predicted '1' (cases) vs '0' (controls) which I could further use to obtain the accuracy measures of the models either with confusionMatrix(predicted, mydata$outcome) or using ModelMetrics library.

So my question is how can I get 4x4 table (predicted vs observed) outcome which I can use to measure the accuracy of my glm() model in predicting the outcome? I will be grateful for any advice, or please let me know if there are better ways of getting the PPV, NPV, sensitivity and specificity. Thank you.

user7757412
  • 35
  • 4

1 Answers1

0

Your glm model is giving probabilities of the two outcomes. Typically, one wants to assign '1' to any event with probability >= . 5 and 0 otherwise. You can do this with round(). In more 'machine-learny' type situations, one might consider different values besides .5. You can use the ifelse() fn to do that. For example, if you want to assign '1' only to cases with probability .7 you could say vals = ifelse(mydata$outcome >.7,1,0 ). Finally, the data you want is usually called a confusion matrix. It can be computed via various packages, but here is a nice solution from a sister site - R: how to make a confusion matrix for a predictive model?

meh
  • 218
  • 1
  • 9
  • Thank you for this. It is very helpful. I have followed your advice and it has worked well. However, my results look rather depressing: only 1 value predicted. I just don't understand what is wrong with my model. I think I might have done something wrong on the way. I will continue investigating. I am really grateful for your help. – user7757412 Sep 28 '18 at 14:21
  • You are doing something wrong. I looked at the head of your data and added '.99' head_pred = c(0.02568802, 0.02979873, 0.01920584, 0.01077031 ,0.01279325, 0.09725329 ,.999) round(head_pred) [1] 0 0 0 0 0 0 1 – meh Sep 28 '18 at 14:50
  • Your first 6 predictions will give zero. Also ifelse(head_pred >.05,1,0) [1] 0 0 0 0 0 1 1 – meh Sep 28 '18 at 14:52