Stochastic Gradient Descent (or SGD) is an algorithm used to find a minima (local or global) of a differentiable function.
Questions tagged [stochastic-gradient]
35 questions
0
votes
1 answer
What is the difference between Stochastic Gradient Descent and LightGBM?
Although I have individually researched these concepts, I am confused on whether one or the other can be chosen for a solution, or can both of these be used simultaneously to improve results? Any guidance you can provide will be much appreciated.
Amie Johnson
- 57
- 1
- 5
0
votes
2 answers
Are weights/biases only updated once per mini-Batch?
Im following a neural networks tutorial, and I have a question about the function that updates the weights.
def update_mini_batch(self, mini_batch, eta):
"""Update the network's weights and biases by applying
gradient descent using…
Psychotechnopath
- 2,471
- 5
- 26
- 47
0
votes
1 answer
Understanding what happens during a single epoch of DBOW
I am using Distributed Bag of Words (DBOW) and I'm curious what happens during a single Epoch? Does DBOW cycle through all documents (aka Batch) or does it cycle through a subset of documents (aka Mini-batch)? In addition, for a given document DBOW…
chethanjjj
- 53
- 5
0
votes
1 answer
stochastic graident for deep learning
I am reading about deep learning concept stochastic gradient. Here in below snap shot I am not understanding what does statement "The general problem with taking a significant step in this direction, however, is that the gradient could be changing…
venkysmarty
- 11,099
- 25
- 101
- 184
-1
votes
1 answer
Stochastic Gradient Descent implementation in Python from scratch. is the implementation correct?
I know this would seem similar to a lot of questions asked previously on the same topic. I have surveyed most of them but they don't quite answer my question. My problem is that my gradient is not converging to optima, it is rather diverging and…
KaranJ
- 48
- 1
- 10