5

I am using the LSTM model that comes by default in tensorflow. I would like to check or to know how to save or show the values of the forget gate in each step, has anyone done this before or at least something similar to this?

Till now I have tried with tf.print but many values appear (even more than the ones I was expecting) I would try plotting something with tensorboard but I think those gates are just variables and not extra layers that I can print (also cause they are inside the TF script)

Any help will be well received

Diego Alejandro Gómez Pardo
  • 391
  • 6
  • 18

1 Answers1

2

If you are using tf.rnn_cell.BasicLSTMCell , the variable you are looking for will have the following suffix in its name : <parent_variable_scope>/BasicLSTMCell/Linear/Matrix . This is a concatenated matrix for all the four gates. Its first dimension matches the sum of the second dimensions of the input matrix and the state matrix (or output of the cell to be exact). The second dimension is 4 times the number of cell size.

The other complementary variable is <parent_variable_scope>/BasicLSTMCell/Linear/Bias that is a vector of the same size as the second dimension of the abovementioned tensor (for obvious reasons).

You can retrieve the parameters for the four gates by using tf.split() along dimension 1. The split matrices would be in the order [input], [new input], [forget], [output]. I am referring to the code here form rnn_cell.py.

Keep in mind that the variable represents the parameters of the Cell and not the output of the respective gates. But with the above info, I am sure you can get that too, if you so desire.

Edit:
Added more specific information about the actual tensors Matrix and Bias

JunkMechanic
  • 438
  • 7
  • 13
  • Hey @JunkMechanic thanks for your message, however I'm very new in tensorflow. Could you tell me how could I establish that scope? I was checking in the TF webpage and I just saw that I could do tf.getVariable and I could use that one with the forget gate, however I didn't see any connection to the variable, more I saw a name of the scope and shape... Is that what I should use? – Diego Alejandro Gómez Pardo Sep 03 '16 at 16:36
  • Yes. You ll have to use `tf.getVariable()`. The `` in my answer is the scope inside which you have called the cell (and not just initialized it). Also keep in mind that you need to set the flag `reuse_variables` for the parent variable scope to be able to retrieve the same tensor. – JunkMechanic Sep 05 '16 at 03:26
  • One way to look for variables is to first list them out using the `log_device_placement` flag in the session config. That will give you an idea of how variables are named in general. – JunkMechanic Sep 05 '16 at 03:31