0

I am using MXNet library in RStudio to train a neural network model.

When training the model using caret, I can tune (among others) the "momentum" parameter. Is this related with the Stochastic Gradient Descent optimizer?

I know that this is the default optimizer when training using "mx.model.FeedForward.create", but what happens when I am using caret:::train??

jmuhlenkamp
  • 2,102
  • 1
  • 14
  • 37
Dimitris Boviatsis
  • 1
  • 1

1 Answers1

0

Momentum is related to SGD and controls how prone your algorithm to change direction of descend. There are several formulas to do that, read more about it here: https://towardsdatascience.com/stochastic-gradient-descent-with-momentum-a84097641a5d

Caret package suppose to be general purpose, so it works with MXNet. When you call cret::train it can accept method parameter. It should be taken from the repository of caret package, which at the moment supports MXNet. See this for an example: https://github.com/topepo/caret/issues/887 from Adam or https://github.com/topepo/caret/blob/master/RegressionTests/Code/mxnet.R for regular SGD.

Sergei
  • 1,617
  • 15
  • 31