0

I am working on an experimental approach to text classification for which I need to use pure Tensorflow (no Keras). As I am implementing Leave-One-Out CV, I need to reset (or reinitialize, call it as you wish) all of my trainable variables in one session - after model learns for several epochs and predicts my test observation it goes to another test observation, however, it should train completely from scratch (without the knowledge received from current test observation which previously used also in learning).

I have done some comprehensive online research, but I did not find anything helpful. Is there any method for that or do you have any suggestions how to modify my algorithm?

Thanks in advance.

Jakub
  • 1
  • 1

1 Answers1

0

You could have two copy of your model : one that contains initial weights and another one that you train from scratch for different samples of your data.

Then when you need to reset, just copy the initial weights to model you train like this :

trainable_model.set_weights(initial_model.get_weights())

More in this answer

samu
  • 1,936
  • 4
  • 22
  • 26
  • Thanks for the suggestion, but how do I do it if I cannot refer to the whole model as above. I have only a set of layers and I can see all the trainable variables with tf.trainable_variables(), but do not know how to copy their states at the beginning. Sorry, I am just a total newbie to tensorflow. – Jakub Mar 07 '20 at 18:05
  • Another idea - could I simply use tf.reset_default_graph() at the end of each iteration? Or close the session and then reset the graph? – Jakub Mar 07 '20 at 18:16
  • Did it work? It would be easier to help if you share your code so far in your question. – samu Mar 08 '20 at 10:22
  • 1
    Sorry that I reply so late. I did not fully understand your suggestion and reseting graph did not work, so the way I finally did it was that for each learning iterations (one left out observation), I am calling tf.global_variables_initializer() and this seems to be working. Do you see any drawbacks in it? – Jakub Mar 21 '20 at 11:23