If I use theano.scan with a truncate_gradient=k(int >0), I can obtain a truncated-BPTT gradient for a recurrent neural network. And that gradient will be calculated for an unrolled for k steps backwards in time RNN(which becomes k-layered feed forward neural network FFNN). Basically(according to theory), that truncated gradient for RNN(calculated by unrolling RNN to k steps backwards in time) should consist of sum of gradients for all k layers of unrolled FFNN(obtained from unrolling RNN to k steps backwards).
Basically, I'm looking how to get each component of that sum in theano... As it seems to me, that you can easily get the resulting truncated grad of an RNN(which is represented by a scan op with truncate_gradient=k)(this functionality works out of the box), but it's quite tricky to get each component of that unrolled sum for truncated gradients for scan op/RNN.
What I already tried:
I looked through the theano scan op internals, especially through the 'grad' method... And it's code is quite complicated.
Also I tried to print graphs for resulting gradients - i tried to print graphs for the same RNN with truncate_gradients=1 and truncate_gradients=10 (using theano.printing.pydotprint) - These graphs were the same :(
Then I used theano.printing.pydotprint with scan_graphs=True That had to print internals of scan. But I failed with such exception:
AttributeError: 'Scan' object has no attribute 'fn'.
Also I googled for these questions - nothing. Asked in theano-users - haven't got any response yet
