1

I have created a Logistic regression model and used it to predict attendance:

LogModel <- glm(formula = Attended ~ City + Duration, 
                family = binomial(logit), data = MyData)
prediction <- predict(LogModel, MyData, type = "response")

What should be the arguments I use in the brierscore() function in order to obtain the brier score?

Jilber Urbina
  • 58,147
  • 10
  • 114
  • 138
user3817648
  • 83
  • 1
  • 7
  • 1
    Not sure where you are getting `brierscore`-function. The print method for `rms::lrm` which does the same modeling as glm(..., family ="binomial") prints a Brier score. Looking at the code it appears there is a `stats` item in the model returned by `lrm`. – IRTFM Aug 05 '14 at 23:41

2 Answers2

5

Note that for a glm fit, fit$residuals will return the working residuals, as opposed to the predicted probabilities. Predicted probabilities can be obtained using residuals(fit,type='response'). See here and here for posts on residual types from a glm fit.

Here is an example of computing the Brier Score using the mtcars dataset:

fit <- glm(am~hp+wt,data=mtcars,family='binomial')
pred.prob <- predict(fit,type='response')
brierScore <- mean((pred.prob-mtcars$am)^2)
# 0.04659236

Here is another post on how to calculate the Brier Score.

Community
  • 1
  • 1
kakarot
  • 376
  • 4
  • 13
2

The Brier score is effectively the mean of the squared residuals. The residuals are stored in every glm model output. So you can just do it by hand:

# Create some data (from ?profile.glm)
ldose <- rep(0:5, 2)
numdead <- c(1, 4, 9, 13, 18, 20, 0, 2, 6, 10, 12, 16)
sex <- factor(rep(c("M", "F"), c(6, 6)))
SF <- cbind(numdead, numalive = 20 - numdead)
# Run a model
budworm.lg0 <- glm(SF ~ sex + ldose - 1, family = binomial)
# Brier score
mean(budworm.lg0$residuals^2)
nograpes
  • 18,623
  • 1
  • 44
  • 67
  • 2
    As noted on the [Wikipedia entry for Brier score](https://en.wikipedia.org/wiki/Brier_score), this isn't the exact formulation used by Brier, but it is the most commonly used. – nograpes Aug 06 '14 at 05:01