Questions tagged [gradient-descent]

Gradient Descent is an algorithm for finding the minimum of a function. It iteratively calculates partial derivatives (gradients) of the function and descends in steps proportional to those partial derivatives. One major application of Gradient Descent is fitting a parameterized model to a set of data: the function to be minimized is an error function for the model.

Wiki:

Gradient descent is a first-order iterative optimization algorithm. It is an optimization algorithm used to find the values of parameters (coefficients) of a function (f) that minimizes a cost function (cost).

To find a local minimum of a function using gradient descent, one takes steps proportional to the negative of the gradient (or of the approximate gradient) of the function at the current point.

Gradient descent is best used when the parameters cannot be calculated analytically (e.g. using linear algebra) and must be searched for by an optimization algorithm.

Gradient descent is also known as steepest descent, or the method of steepest descent.

Tag usage:

Questions on gradient-descent should be about implementation and programming problems, not about the theoretical properties of the optimization algorithm. Consider whether your question might be better suited to Cross Validated, the StackExchange site for statistics, machine learning and data analysis.

Read more:

1428 questions

-1

votes

1 answer

optimal values of a,b,c,d,e based on data passed to a function with known ground truth values

I have a function which takes several parameters - a,b,c,d,e and the returns the computed value of z. I also have the ground truth value of z and I would like to compute the optimal parameters of a,b,c,d,e which would minimize the error between the…

asked Oct 24 '18 at 10:55

Ash

-1

votes

2 answers

Gradient Descent Algorithm, Gradient stepping function

I am trying to understand the Gradient Descent Algorithm. The code here should choose a more optimal line of best fit, given another line of best fit. The function takes the current line-of-best-fit's slope and y-intercept as inputs, as well as a…

python machine-learning artificial-intelligence gradient-descent

asked Jul 14 '18 at 03:23

ParksideAdrian

-1

votes

1 answer

In multi-class logistic regression, does SGD one training example update all the weights?

In multi-class logistic regression, lets say we use softmax and cross entropy. Does SGD one training example update all the weights or only a portion of the weights which are associated to the label ? For example, the label is one-hot [0,0,1] Does…

machine-learning logistic-regression data-science gradient-descent

asked Jun 18 '18 at 04:11

Ancalagon BerenLuthien

1,154
6
20
29

-1

votes

1 answer

Using conjugate gradient in pose-estimation

graphical representation of my pipeline I'm trying to get a pose estimation of a face captured from a video feed. I use a face alignement algorithm of tracking.js. This give me points which I then use to try to estimate the position of my face in…

javascript algorithm machine-learning gradient-descent

asked Jun 14 '18 at 16:06

Gabriel Bouvier-Denoix

-1

votes

1 answer

Accuracy on training data in Gradient boosting classifier- scikit

I am training GBC. It is multi class classifier with 12 classes of outputs. My issue is I am not getting 100% accuracy when i predict on the train data. In fact, misprediction happens on dominant set of classes. (my input is imbanalanced and i do…

python machine-learning scikit-learn gradient-descent boosting

asked Jun 14 '18 at 02:04

sranga

-1

votes

1 answer

How to get the minimum cross section of a 3D volumn

Now there is a 3D volume stored in 3d array, a point is given in the 3d volume. How to calculate the minimum cross section of the volume through the point?

python image-processing 3d gradient-descent

asked May 30 '18 at 06:37

Silver0427

-1

votes

2 answers

Why that I have written for Andrew Ng's course not accepted?

Andrew Ng's course in Coursera, which Stanford's Machine Learning course, features programming assignments that deal with implementing the algorithms taught in class. The goal of this assignment is to implement linear regression through gradient…

octave gradient-descent

asked May 26 '18 at 09:06

user4417148

-1

votes

1 answer

Relationship logistic regression and Stochastic gradient descent In Formula

Considering in Formula term, from my opinion that SGD apply to final result of Logistic regression. Not sure whether correct or not just wondering the relationship between stochastic gradient descent and logistic regression. I m guessing it works…

python machine-learning logistic-regression gradient-descent

asked May 18 '18 at 14:22

Tony G

-1

votes

2 answers

How to implement the remaining 'for loop' in this neural network in TensorFlow

I am trying to get a neural network going in TensforFlow. The dataset is simply length and width of a flower petal and the output can be either 1/0 depending on type: x = [[3,1.5], [2,1], [4,1.5], [3,1], [3.5,0.5], …

python for-loop tensorflow neural-network gradient-descent

asked Apr 30 '18 at 13:36

Himansu Odedra

-1

votes

1 answer

what does parameters = int(theta.ravel().shape[1]) mean?

Can someone explain that code for me? def gradientDescent(X, y, theta, alpha, iters): temp = np.matrix(np.zeros(theta.shape)) parameters = int(theta.ravel().shape[1]) cost = np.zeros(iters) for i in range(iters): error = (X *…

python numpy linear-regression gradient-descent

asked Apr 15 '18 at 23:59

BorkoP

-1

votes

1 answer

TypeError: only length-1 arrays can be converted to Python scalars Dot Product

Writing this algorithm for my final year project. Debugged a few, but stuck on this. Tried changing the float method but nothing really changed. ----> 8 hypothesis = np.dot(float(x), theta) TypeError: only length-1 arrays can be converted to…

python pandas numpy machine-learning gradient-descent

asked Apr 04 '18 at 18:58

capncook

-1

votes

2 answers

Weights becoming "NaN" in implementation of Neural Networks

I am trying to implement Neural Networks for classifcation having 5 hidden layers, and with softmax cross entropy in the output layer. The implementation is in JAVA. For optimization, I have used MiniBatch gradient descent(Batch size=100, learning…

machine-learning neural-network gradient-descent softmax multilabel-classification

asked Apr 04 '18 at 12:03

204

-1

votes

1 answer

Gradient descent - can I draw function that I will minimize? Linear regression

I'm new in machine learning. I started from linear regression with gradient descent. I have python code for this and I understad this way. My question is: Gradient descent algorithm minimize function, can I plot this function? I want to see what the…

python machine-learning linear-regression gradient-descent

asked Mar 20 '18 at 21:25

lukassz

3,135
7
32
72

-1

votes

1 answer

How backpropagation through gradient descent represents the error after each forward pass

In Neural NEtwork Multilayer Perceptron, I understand that the main difference between Stochastic Gradient Descent (SGD) vs Gradient Descent (GD) lies in the way of how many samples are chosen while training. That is, SGD iteratively chooses one…

machine-learning scikit-learn neural-network gradient-descent loss-function

asked Dec 09 '17 at 13:57

Dave

-1

votes

1 answer

Decrement learning rate in error back propagation algorithm

This is more or less general question, in my implementation of backpropagation algorithm, I start from some "big" learning rate, and then decrease it after I see the error started to grow, instead of narrowing down. I am able to do this rate…

machine-learning neural-network backpropagation gradient-descent supervised-learning

asked Oct 22 '17 at 08:52

Павел

Prev 1 2 3

…

95 96 Next