Stack Overflow
Stack Overflow

Questions tagged [back-propagation-through-time]

7 questions
2
votes
1 answer

Multiple unmatched matrices in backpropagation through time

I am going to implement binary addition by Recurrent Neural Network (RNN) as a sample. I have coped with an issue to implement it by Python, so I decided to share my problem in there to come up with ideas to fix it. As can be seen in my notebook…
Ali Soltani
  • 9,589
  • 5
  • 30
  • 55
1
vote
0 answers

Transferring the hidden state of a RNN to another RNN

I am using Reinforcement Learning to teach an AI an Austrian Card Game with imperfect information called Schnapsen. For different states of the game, I have different neural networks (which use different features) that calculate the value/policy. I…
Babypopo
  • 131
  • 1
  • 7
0
votes
0 answers

What are the potential reasons for slow performance in my backpropagation through time implementation for a recurrent neural network?

I decided to code backpropagation through time algo in order to train a rnn without lstm and without bias. I am using cross entropy loss function and tanh activation fn at hidden layer and softmax activation function at output layer. The rnn…
gurjyot
  • 1
  • 2
0
votes
0 answers

How to combine A2C with BPTT?

I'm having a little difficulty understanding how I can apply backpropagation through time to the A2C method, or any reinforcement learning method for that matter. As I understand it, BPTT conceptually unrolls a recurrent network and performs a…
Telf
  • 1
0
votes
0 answers

LSTM RNN Walking Forward for timeseries

I am confused in using walking forward validation. I have found several pages that have different or not clear wording on "validation" and "test". Basically, when I apply it to time series forecasting, I divide the data into training, validation and…
prgtttt
  • 1
  • 1
0
votes
1 answer

Automatic differentiation for incrementally updated states

Can automatic differentiation (e.g. tensorflow/pytorch) intelligently backpropagate through a [L2] neuron (of size batchSize) that was incrementally updated (addition of existing activated batch elements and newly activated batch elements) over time…
user2585501
  • 596
  • 4
  • 17
0
votes
2 answers

What is wrong with my BPTT Implementation?

I tried to implement Backpropagation through time manually, but in the end the network isn't converging. I tried looking around on the net for descriptions and courses on BPTT, and the code does everything accordingly: Forward propagation Error…
Dávid Tóth
  • 2,788
  • 1
  • 21
  • 46