19

The program I'm writing involves switching between models during run-time.

I am currently using Saver to save/load models from the disk as specified here: https://www.tensorflow.org/api_docs/python/state_ops/saving_and_restoring_variables#Saver.

The models are fairly small and can be stored in memory, so I was wondering if anyone knows of a way to store and restore these models in-memory instead of saving them to disk.

I tried to modify the tensorflow source to save the model to memory however gen_io_ops seems to be generated during compile time. Another possible way is to use memory mapped files. Does anyone know of an easier way?

jmd_dk
  • 12,125
  • 9
  • 63
  • 94
anubhavashok
  • 532
  • 3
  • 15

1 Answers1

1

I would just have two different sessions with their own computation graphs. Alternatively, you could just duplicate the computation graph (two copies of the variables, operations, etc) in the same session. Then you would call sess.run(comp1 if useCompOne else comp2), however you'd like to set it up.

Jacob Holloway
  • 887
  • 8
  • 24
  • 1
    This approach (assuming both sessions are active) will keep the variables where they are, i.e., in GPU memory; if the GPU memory is a bottleneck (which it often is) the OP would need a solution to move the inactive session data from GPU to the cheaper/larger system RAM. – Peteris Jan 10 '17 at 23:49
  • 1
    Very true. I was working off of his comment: "The models are fairly small and can be stored in memory". I suppose you could have a copy of both sets on the RAM, and a computation set on the GPU memory, and then write a 'transfer' op to copy the current values from a gpu to ram and viceversa if that that was something that needed, but I'd look into exploring other options once they no longer fit in the memory together. – Jacob Holloway Jan 10 '17 at 23:50