7

Keras assigns incrementing ID numbers to layers of the same type, e.g. max_pooling1d_7, max_pooling1d_8, max_pooling1d_9,etc. Each iteration of my code constructs a new model, starting with model = Sequential() and then adding layers via model.add(). Even though each cycle creates a new Sequential object, the layer ID numbers continue incrementing from the previous cycle. Since my process is long-running these ID numbers can grow very large. I am concerned that this could cause some problem. Why are the IDs not reset by model = Sequential()? Is there a way to reset them? After each cycle I have no use for the layer ID numbers and can discard them, but how? I am using the Tensorflow backend.

Ron Cohen
  • 2,815
  • 5
  • 30
  • 45

4 Answers4

7

The solution, from Attempting to reset tensorflow graph when using keras, failing:

from keras import backend as K
K.clear_session()
Ron Cohen
  • 2,815
  • 5
  • 30
  • 45
  • This does not work. In tensorflow 2.0. creating a new model after reseting still has numbers in the layers that are larger and larger. – mathtick Apr 10 '19 at 16:41
1

Do you use jupyter notebooks? It seems like while you're rebuilding your model your tensorlow session won't restart. Because keras refers to tensorflow graphs per name it's necessary, that the counting continues.

So if you don't want to restartthe session you're fine. However this also means, that the tensorflow session gets bigger and bigger, so restarting the session might be the desired approach. For this restart the complete program/kernel.

dennis-w
  • 2,166
  • 1
  • 13
  • 23
  • I am not using jupyter, just python2.7 + Keras. I am evolving my networks via genetic algorithm. I keep a lot of data in memory between each cycle (=generation) so I cannot restart the program/kernel between generations unless I swap the data out to disk, which would be quite slow. – Ron Cohen Mar 06 '18 at 15:57
  • Well there are some function in tensorflow to reset the session/ graph. Last time I heard from them, they didn‘t work properly. But maybe this is fixed now. So searching for reset tensorflow session could help your problem. (I also can look into it tomorrow) – dennis-w Mar 06 '18 at 16:25
  • 2
    I just found this and it works: https://stackoverflow.com/questions/45063602/attempting-to-reset-tensorflow-graph-when-using-keras-failing – Ron Cohen Mar 06 '18 at 16:56
0

Each iteration shouldn't construct a new model. Training should go through in same model. Maybe post your code to see what might goes wrong.

GUZi
  • 306
  • 3
  • 7
  • No, generation of new models is completely intentional. My models evolve, based on their performance, via genetic algorithm. – Ron Cohen Mar 06 '18 at 15:51
0

Add

tf.keras.backend.clear_session()

as in https://www.tensorflow.org/api_docs/python/tf/keras/backend/clear_session

Maged
  • 818
  • 1
  • 8
  • 17