0

How can I specify weight decay in a model fit by the mlogit?

The multinom() function of nnet allows you to specify weight decay for the model that is being fit, and mlogit uses this function behind the scenes to fit its models so I imagine that it should be possible to pass the decay argument to multinom, but have not so far found a way to do this.

So far I have attempted to simply pass a value in the model formula, like this.

library(mlogit)

set.seed(1)

data("Fishing", package = "mlogit")
Fishing$wts <- runif(nrow(Fishing)) #for some weights
Fish <- mlogit.data(Fishing, varying = c(2:9), shape = "wide", choice = "mode")    

fit1 <- mlogit(mode ~ 0 | income, data = Fish, weights = wts, decay = .01)
fit2 <- mlogit(mode ~ 0 | income, data = Fish, weights = wts)

But the output is exactly the same:

identical(logLik(fit1), logLik(fit2))
[1] TRUE
double-beep
  • 5,031
  • 17
  • 33
  • 41
Johan Larsson
  • 3,496
  • 18
  • 34

1 Answers1

2

mlogit() and nnet::multinom() both fit multinomial logistic models (predicting probability of class membership for multiple classes) but they use different algorithms to fit the model. nnet::multinom() uses a neural network to fit the model and mlogit() uses maximum likelihood.

Weight decay is a parameter for neural networks and is not applicable to maximum likelihood.

The effect of weight decay is keep the weights in the neural network from getting too large by penalizing larger weights during the weight update step of the fitting algorithm. This helps to prevent over-fitting and hopefully creates a more general model.

Consider using the pmlr function in the pmlr package. This function implements a "Penalized maximum likelihood estimation for multinomial logistic regression" when called with the default function parameter penalized = TRUE.

brandco
  • 310
  • 2
  • 6
  • Very interesting, thanks! I had missed that package completely though I think I might still prefer `multinom` since it can be used with `caret`! – Johan Larsson Aug 05 '15 at 14:19