2

I am wondering what exactly is saved when I use a tf.train.Saver() to save my model after every training epoch. The file seems kind of large compared to what I am used to with Keras models. Right now my RNN takes up 900 MB at each save. Is there any way to tell the saver to only save the trainable parameters? I would also like a way to save only part of the model. I know I can just get the variables I define and save them using the numpy format but when I use the RNN classes I don't directly have access to their weights and I looked through the code and there is nothing like get_weights that I can see.

chasep255
  • 11,745
  • 8
  • 58
  • 115

2 Answers2

6

You can provide a list of variables to save in the Saver constructor, ie saver=tf.train.Saver(var_list=tf.trainable_variables())

Yaroslav Bulatov
  • 57,332
  • 22
  • 139
  • 197
  • 1
    This is incomplete as some variables are not trainable but are still saved by the Saver. – Conchylicultor Jan 23 '18 at 01:44
  • Hi, I wonder to know, if the network contains BN layers, what should I do? Because the mean and variance are variables, but they are not trainable. If I only save tf.tf.trainable_variables(), when I restore the model, tensorflow can't find the BN mean parameters...... – Ariel Nov 07 '18 at 03:37
4

It will save all variables._all_saveable_objects() by default, if Saver does not specify var_list.

That is, Saver will save all global variables and saveable variables by default.

def _all_saveable_objects():
  """Returns all variables and `SaveableObject`s that must be checkpointed.

  Returns:
    A list of `Variable` and `SaveableObject` to be checkpointed
  """
  # TODO(andreasst): make this function public once things are settled.
  return (ops.get_collection(ops.GraphKeys.GLOBAL_VARIABLES) +
          ops.get_collection(ops.GraphKeys.SAVEABLE_OBJECTS))
Guangcong Liu
  • 805
  • 1
  • 8
  • 6