What is the difference between tf.initialize_all_variables() and tf.initialize_local_variables()?

Question

I am reviewing the code in this example: fully_connected_reader.py

I am confused with Line 147 and 148:

init_op = tf.group(tf.initialize_all_variables(),
                   tf.initialize_local_variables())

I don't know which variables are all variables and which are local variables. Any ideas?

score 11 · Answer 1 · edited Apr 07 '17 at 04:26

11

tf.initialize_all_variables() is a shortcut to tf.initialize_variables(tf.all_variables()), tf.initialize_local_variables() is a shortcut to tf.initialize_variables(tf.local_variables()), which initializes variables in GraphKeys.VARIABLES and GraphKeys.LOCAL_VARIABLE collections, respectively.

Variables in GraphKeys.LOCAL_VARIABLES collection are variables that are added to the graph, but not saved or restored (source).

tf.Variable() by default adds a new variable to GraphKeys.VARIABLE collection, which can be controlled by collections= argument.

edited Apr 07 '17 at 04:26

David Parks

30,789
47
185
328

answered Oct 24 '16 at 16:37

M U

111
2

Thanks for your answer. How can I identify whether there is any local variables to initialize? – C. Wang Oct 25 '16 at 07:12
Should be able to graph them and print them out, this page has the reference you need https://www.tensorflow.org/api_docs/python/tf/GraphKeys – David Parks Apr 07 '17 at 04:29

score 5 · Answer 2 · edited May 23 '17 at 12:17

A local variable in TF is any variable which was created with collections=[tf.GraphKeys.LOCAL_VARIABLES]. For example:

e = tf.Variable(6, name='var_e', collections=[tf.GraphKeys.LOCAL_VARIABLES])

LOCAL_VARIABLES: the subset of Variable objects that are local to each machine. Usually used for temporarily variables, like counters. Note: use tf.contrib.framework.local_variable to add to this collection.

They are usually not saved/restored to checkpoint and used for temporary or intermediate values. For a more detailed answer, take a look here.

A global variable is mostly every other variable initialized by you.

In a new version of TF you should use tf.global_variables_initializer(), tf.local_variables_initializer(), because the previous functions were deprecated.

score 0 · Answer 3 · answered Jul 17 '17 at 07:13

GLOBAL_VARIABLES

Key to collect Variable objects that are global (shared across machines). Default collection for all variables, except local ones.

LOCAL_VARIABLES

Key to collect local variables that are local to the machine and are not saved/restored.

What is the difference between tf.initialize_all_variables() and tf.initialize_local_variables()?

3 Answers3

Linked