-1

Considering in Formula term, from my opinion that SGD apply to final result of Logistic regression. Not sure whether correct or not

just wondering the relationship between stochastic gradient descent and logistic regression. I m guessing it works similar to Neuron network where it calculates how to improve weight (apply the chain rule) .

In other word, after the LR formula is calculated then it applies chain rule on top of it. to get a better weight to and keep it in loop unit LR loss rate close to zero , is this correct? Thx

for example, after finish the LR calculation,

then apply SGD(LR) -> get updated weight -> perform LR again in loop until SGD is satisfied

Tony G
  • 55
  • 1
  • 5

1 Answers1

1

Logistic regression is a way to model a discrete system using a logistic function (or similar variant). That is, a system who's output has a finite number of possible values. You can think of it as a kind of classification algorithm (though this description can be dangerous, since classification is technically different than regression), which maps a set of inputs to a finite set of outputs.

Stochastic gradient descent is a variant of gradient descent (or batch gradient descent) optimization algorithm. Instead of using all (or a "batch" of) training data simultaneously (can be very computation/memory expensive), it uses an iterative approximation for finding the minimum of a function across N-dimensional input space.

Stochastic gradient descent can be used to build a logistic regression model similar to how it can be used to build a linear regression model. The model itself is independent of the optimization algorithm used to train it. While stochastic gradient descent is commonly used as a training algorithm, it isn't the ONLY option.

jmkmay
  • 1,441
  • 11
  • 21