Stack Overflow
Stack Overflow

Questions tagged [backpropagation]

Backpropagation is a method of the gradient computation, often used in artificial neural networks to perform gradient descent.

Backpropagation is a method of the gradient computation, often used in artificial neural networks to perform gradient descent. It led to a “renaissance” in the field of artificial neural network research.

In most cases, it requires a teacher that knows, or can calculate, the desired output for any input in the training set. The term is an abbreviation for "backward propagation of errors".

1267 questions
9
votes
1 answer

Can I (selectively) invert Theano gradients during backpropagation?

I'm keen to make use of the architecture proposed in the recent paper "Unsupervised Domain Adaptation by Backpropagation" in the Lasagne/Theano framework. The thing about this paper that makes it a bit unusual is that it incorporates a 'gradient…
Bill Cheatham
  • 11,396
  • 17
  • 69
  • 104
9
votes
3 answers

Backpropogation activation derivative

I've implemented backpropagation as explained in this video. https://class.coursera.org/ml-005/lecture/51 This seems to have worked successfully, passing gradient checking and allowing me to train on MNIST digits. However, I've noticed most other…
Kieren Anderson
  • 508
  • 3
  • 16
9
votes
1 answer

Why does this backpropagation implementation fail to train weights correctly?

I've written the following backpropagation routine for a neural network, using the code here as an example. The issue I'm facing is confusing me, and has pushed my debugging skills to their limit. The problem I am facing is rather simple: as the…
user1131435
8
votes
2 answers

Pytorch: RuntimeError: expected dtype Float but got dtype Long

I encounter this weird error when building a simple NN in Pytorch. I dont understand this error and why this consern Long and Float datatype in backward function. Anyone encounter this before? Thanks for any help. Traceback (most recent call last): …
Yun Liu
  • 81
  • 1
  • 1
  • 2
8
votes
3 answers

Gradcam with guided backprop for transfer learning in Tensorflow 2.0

I get an error using gradient visualization with transfer learning in TF 2.0. The gradient visualization works on a model that does not use transfer learning. When I run my code I get the error: assert str(id(x)) in tensor_dict, 'Could not…
aveevu
  • 767
  • 6
  • 15
8
votes
1 answer

How is a multiple-outputs deep learning model trained?

I think I do not understand the multiple-output networks. Althrough i understand how the implementation is made and i succesfully trained one model like this, i don't understand how a multiple-outputs deep learning network is trained. I mean, what…
Baptiste Pouthier
  • 573
  • 3
  • 22
8
votes
1 answer

Backpropagation algorithm giving bad results

I'm trying to tackle the classic handwritten digit recognition problem with a feed forward neural network and backpropagation, using the MNIST dataset. I'm using Michael Nielsen's book to learn the essentials and 3Blue1Brown's youtube video for the…
H. Saleh
  • 554
  • 1
  • 5
  • 19
8
votes
5 answers

can't find the inplace operation: one of the variables needed for gradient computation has been modified by an inplace operation

I am trying to compute a loss on the jacobian of the network (i.e. to perform double backprop), and I get the following error: RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation I can't find…
Einav
  • 91
  • 1
  • 1
  • 4
8
votes
1 answer

TensorFlow average gradients over several batches

This is a possible duplicate of Tensorflow: How to get gradients per instance in a batch?. I ask it anyway, because there has not been a satisfying answer and the goal here is a bit different. I have a very big network that I can fit on my GPU but…
niko
  • 1,128
  • 1
  • 11
  • 25
8
votes
2 answers

Guided Back-propagation in TensorFlow

I would like to implement in TensorFlow the technique of "Guided back-propagation" introduced in this Paper and which is described in this recipe . Computationally that means that when I compute the gradient e.g., of the input wrt. the output of…
Peter
  • 1,541
  • 3
  • 11
  • 11
8
votes
1 answer

XOR Neural Network Converges to 0.5

I've implemented the following neural network to solve the XOR problem in Python. My neural network consists of an input layer of 2 neurons, 1 hidden layer of 2 neurons and an output layer of 1 neuron. I am using the Sigmoid function as the…
sam
  • 417
  • 6
  • 16
8
votes
1 answer

Neural network backpropagation algorithm not working in Python

I am writing a neural network in Python, following the example here. It seems that the backpropagation algorithm isn't working, given that the neural network fails to produce the right value (within a margin of error) after being trained 10 thousand…
user76284
  • 1,269
  • 14
  • 29
8
votes
1 answer

Does Theano do automatic unfolding for BPTT?

I am implementing an RNN in Theano and I have difficulties training it. It doesn't even come near to memorising the training corpus. My mistake is most likely caused by me not understanding exactly how Theano copes with backpropagation through time.…
John Jaques
  • 560
  • 4
  • 17
8
votes
4 answers

Backpropagation for rectified linear unit activation with cross entropy error

I'm trying to implement gradient calculation for neural networks using backpropagation. I cannot get it to work with cross entropy error and rectified linear unit (ReLU) as activation. I managed to get my implementation working for squared error…
Pr1mer
  • 81
  • 1
  • 1
  • 3
7
votes
1 answer

Why is a bias neuron necessary for a backpropagating neural network that recognizes the XOR operator?

I posted a question yesterday regarding issues that I was having with my backpropagating neural network for the XOR operator. I did a little more work and realized that it may have to do with not having a bias neuron. My question is, what is the…
Vivin Paliath
  • 94,126
  • 40
  • 223
  • 295
1 2 3
…
84 85