Stack Overflow
Stack Overflow

Questions tagged [backpropagation]

Backpropagation is a method of the gradient computation, often used in artificial neural networks to perform gradient descent.

Backpropagation is a method of the gradient computation, often used in artificial neural networks to perform gradient descent. It led to a “renaissance” in the field of artificial neural network research.

In most cases, it requires a teacher that knows, or can calculate, the desired output for any input in the training set. The term is an abbreviation for "backward propagation of errors".

1267 questions
13
votes
1 answer

The concept of straight through estimator (STE)

I have seen straight through estimator (STE) in many Neural Network related papers e.g. this and this. But I cannot understand the concept. I wonder if anyone could explain STE or refer me to a simple resource?
Amir
  • 141
  • 1
  • 1
  • 7
13
votes
3 answers

Neural network in Javascript not learning properly

I've tried to rewrite neural network found here to javascript. My javascript code looks like this. function NeuralFactor(weight) { var self = this; this.weight = weight; this.delta = 0; } function Sigmoid(value) { return 1 / (1 +…
Łukasz W.
  • 9,538
  • 5
  • 38
  • 63
12
votes
2 answers

Difference between autograd.grad and autograd.backward?

Suppose I have my custom loss function and I want to fit the solution of some differential equation with help of my neural network. So in each forward pass, I am calculating the output of my neural net and then calculating the loss by taking the MSE…
unstableEquilibrium
  • 157
  • 1
  • 7
12
votes
1 answer

PyTorch : predict single example

Following the example from: https://github.com/jcjohnson/pytorch-examples This code trains successfully: # Code in file tensor/two_layer_net_tensor.py import torch device = torch.device('cpu') # device = torch.device('cuda') # Uncomment this to…
blue-sky
  • 51,962
  • 152
  • 427
  • 752
12
votes
2 answers

Debugging a Neural Network

TLDR I have been trying to fit a simple neural network on MNIST, and it works for a small debugging setup, but when I bring it over to a subset of MNIST, it trains super fast and the gradient is close to 0 very quickly, but then it outputs the same…
olliejday
  • 196
  • 8
12
votes
2 answers

LSTM RNN Backpropagation

Could someone give a clear explanation of backpropagation for LSTM RNNs? This is the type structure I am working with. My question is not posed at what is back propagation, I understand it is a reverse order method of calculating the error of the…
Jjoseph
  • 206
  • 2
  • 9
11
votes
4 answers

Training feedforward neural network for OCR

Currently I'm learning about neural networks and I'm trying to create an application that can be trained to recognize handwritten characters. For this problem I use a feed-forward neural network and it seems to work when I train it to recognize 1,…
Marnix v. R.
  • 1,638
  • 4
  • 22
  • 33
11
votes
4 answers

How does a back-propagation training algorithm work?

I've been trying to learn how back-propagation works with neural networks, but yet to find a good explanation from a less technical aspect. How does back-propagation work? How does it learn from a training dataset provided? I will have to code…
unleashed
  • 915
  • 2
  • 16
  • 35
11
votes
1 answer

ND Convolution Backprogation

For my education, I am trying to implement an N-dimensional convolutional layer in a convolutional neural network. I would like to implement a backpropagation function. However, I am not sure of the most efficient way of doing so. At present, I am…
Jack Rolph
  • 587
  • 5
  • 15
11
votes
4 answers

Are there alternatives to backpropagation?

I know a neural network can be trained using gradient descent and I understand how it works. Recently, I stumbled upon other training algorithms: conjugate gradient and quasi-Newton algorithms. I tried to understand how they work but the only good…
Nope
  • 153
  • 1
  • 6
11
votes
1 answer

How to build a multiple input graph with tensor flow?

Is it possible to define a TensorFlow graph with more than one input? For instance, I want to give the graph two images and one text, each one is processed by a bunch of layers with a fc layer at the end. Then there is a node that computes a loss…
user2614596
  • 630
  • 2
  • 11
  • 30
11
votes
1 answer

Updating weights in backpropagation algorithm

I think I've understood each step of backpropagation algorithm but the most important one. How do weights get updated? Like at the end of this tutorial? http://home.agh.edu.pl/~vlsi/AI/backp_t_en/backprop.html
Snate
  • 195
  • 1
  • 1
  • 9
11
votes
3 answers

Backpropagation algorithm through cross-channel local response normalization (LRN) layer

I am working on replicating a neural network. I'm trying to get an understanding of how the standard layer types work. In particular, I'm having trouble finding a description anywhere of how cross-channel normalisation layers behave on the…
user1488804
  • 840
  • 1
  • 13
  • 27
11
votes
3 answers

Weather forecast using a neural network

I am trying to write a program for weather forecasting using backpropagation. I am a beginner in this field. I have historical data with different parameters like temperature, humidity, wind speed, rainfall etc. I am confused about how to provide…
amey
  • 127
  • 1
  • 3
11
votes
2 answers

Neural Network learning rate and batch weight update

I have programmed a Neural Network in Java and am now working on the back-propagation algorithm. I've read that batch updates of the weights will cause a more stable gradient search instead of a online weight update. As a test I've created a time…
avanwieringen
  • 2,404
  • 3
  • 21
  • 28
1 2
3
…
84 85