Using Pearson Correlation instead of accuracy in tensorflow to report performance

Question

In my TF code I defined the following in my model:

with tf.name_scope("loss"):
      self.loss = self.contrastive_loss(self.input_y,self.distance, batch_size) 
with tf.name_scope("accuracy"):
      correct_predictions = tf.equal(self.distance, self.input_y)
      self.accuracy=tf.reduce_mean(tf.cast(correct_predictions, "float"), name="accuracy")

But I realised that it's more appropriate to use pearson correlation in my task. So, I changed it to this:

with tf.name_scope("loss"):
      self.loss = self.contrastive_loss(self.input_y,self.distance, batch_size) 
with tf.name_scope("pearson"):
      self.pearson = tf.contrib.metrics.streaming_pearson_correlation(self.distance, self.input_y, name="pearson")

I call my model in the computational graph and in the training loop I have this evaluation code:

acc = dev_step(dev_x1_batch, dev_x2_batch, dev_y_batch, X)

which calls this method:

def dev_step(x1_batch, x2_batch, y_batch, X):
    """
    A single training step
    """ 
    if random()>0.5:
        feed_dict = {
                         siameseModel.input_x1: x1_batch,
                         siameseModel.input_x2: x2_batch,
                         siameseModel.input_y: y_batch,
                         siameseModel.dropout_keep_prob: FLAGS.dropout_keep_prob,
                         siameseModel.embedding_placeholder: X
        }
    else:
        feed_dict = {
                         siameseModel.input_x1: x2_batch,
                         siameseModel.input_x2: x1_batch,
                         siameseModel.input_y: y_batch,
                         siameseModel.dropout_keep_prob: FLAGS.dropout_keep_prob,
                         siameseModel.embedding_placeholder: X
        }
    step, _, loss, pearson, dist, out1, out2 = sess.run([global_step, siameseModel.embedding_init, siameseModel.loss, siameseModel.pearson, siameseModel.distance, siameseModel.out1, siameseModel.out2],  feed_dict)
    print("DEV {}: step {}, loss {:g}, acc {:g}".format(time_str, step, loss, pearson))
    return pearson

All this I do within a TF session:

sess = tf.Session(config=session_conf)
....code to create instance of model, definition of training procedure
sess.run(tf.initialize_all_variables())
...training loop

But I get this error:

tensorflow/core/framework/op_kernel.cc:968] Failed precondition: Attempting to use uninitialized value pearson/pearson/covariance/count
 [[Node: pearson/pearson/covariance/count/read = Identity[T=DT_FLOAT, _class=["loc:@pearson/pearson/covariance/count"], _device="/job:localhost/replica:0/task:0/cpu:0"](pearson/pearson/covariance/count)]]
tensorflow/core/framework/op_kernel.cc:968] Failed precondition: Attempting to use uninitialized value pearson/pearson/covariance/count
 [[Node: pearson/pearson/covariance/count/read = Identity[T=DT_FLOAT, _class=["loc:@pearson/pearson/covariance/count"], _device="/job:localhost/replica:0/task:0/cpu:0"](pearson/pearson/covariance/count)]]
tensorflow/core/framework/op_kernel.cc:968] Failed precondition: Attempting to use uninitialized value pearson/pearson/covariance/count
 [[Node: pearson/pearson/covariance/count/read = Identity[T=DT_FLOAT, _class=["loc:@pearson/pearson/covariance/count"], _device="/job:localhost/replica:0/task:0/cpu:0"](pearson/pearson/covariance/count)]]
tensorflow/core/framework/op_kernel.cc:968] Failed precondition: Attempting to use uninitialized value pearson/pearson/covariance/count
 [[Node: pearson/pearson/covariance/count/read = Identity[T=DT_FLOAT, _class=["loc:@pearson/pearson/covariance/count"], _device="/job:localhost/replica:0/task:0/cpu:0"](pearson/pearson/covariance/count)]]
tensorflow/core/framework/op_kernel.cc:968] Failed precondition: Attempting to use uninitialized value pearson/pearson/covariance/count
 [[Node: pearson/pearson/covariance/count/read = Identity[T=DT_FLOAT, _class=["loc:@pearson/pearson/covariance/count"], _device="/job:localhost/replica:0/task:0/cpu:0"](pearson/pearson/covariance/count)]]
tensorflow/core/framework/op_kernel.cc:968] Failed precondition: Attempting to use uninitialized value pearson/pearson/covariance/count
 [[Node: pearson/pearson/covariance/count/read = Identity[T=DT_FLOAT, _class=["loc:@pearson/pearson/covariance/count"], _device="/job:localhost/replica:0/task:0/cpu:0"](pearson/pearson/covariance/count)]]
Traceback (most recent call last):
  File "train.py", line 260, in <module>
acc = dev_step(dev_x1_batch, dev_x2_batch, dev_y_batch, X)
  File "train.py", line 207, in dev_step
step, _, loss, pearson, dist, out1, out2 = sess.run([global_step, siameseModel.embedding_init, siameseModel.loss, siameseModel.pearson, siameseModel.distance, siameseModel.out1, siameseModel.out2],  feed_dict)
  File "/Users/kurt/anaconda2/envs/tensorflow/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 717, in run
run_metadata_ptr)
  File "/Users/kurt/anaconda2/envs/tensorflow/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 915, in _run
feed_dict_string, options, run_metadata)
  File "/Users/kurt/anaconda2/envs/tensorflow/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 965, in _do_run
target_list, options, run_metadata)
  File "/Users/kurt/anaconda2/envs/tensorflow/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 985, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors.FailedPreconditionError: Attempting to use uninitialized value pearson/pearson/covariance/count
 [[Node: pearson/pearson/covariance/count/read = Identity[T=DT_FLOAT, _class=["loc:@pearson/pearson/covariance/count"], _device="/job:localhost/replica:0/task:0/cpu:0"](pearson/pearson/covariance/count)]]

Caused by op u'pearson/pearson/covariance/count/read', defined at:
  File "train.py", line 97, in <module>
batch_size=FLAGS.batch_size)
  File "filepath/", line 89, in __init__
self.pearson = tf.contrib.metrics.streaming_pearson_correlation(self.distance, self.input_y, name="pearson")
  File "/Users/kurt/anaconda2/envs/tensorflow/lib/python2.7/site-packages/tensorflow/contrib/metrics/python/ops/metric_ops.py", line 2490, in streaming_pearson_correlation
predictions, labels, weights=weights, name='covariance')
  File "/Users/kurt/anaconda2/envs/tensorflow/lib/python2.7/site-packages/tensorflow/contrib/metrics/python/ops/metric_ops.py", line 2367, in streaming_covariance
count = _create_local('count', [])
  File "/Users/kurt/anaconda2/envs/tensorflow/lib/python2.7/site-packages/tensorflow/contrib/metrics/python/ops/metric_ops.py", line 135, in _create_local
collections=collections)
  File "/Users/kurt/anaconda2/envs/tensorflow/lib/python2.7/site-packages/tensorflow/python/ops/variables.py", line 215, in __init__
dtype=dtype)
  File "/Users/kurt/anaconda2/envs/tensorflow/lib/python2.7/site-packages/tensorflow/python/ops/variables.py", line 327, in _init_from_args
self._snapshot = array_ops.identity(self._variable, name="read")
  File "/Users/kurt/anaconda2/envs/tensorflow/lib/python2.7/site-packages/tensorflow/python/ops/gen_array_ops.py", line 1128, in identity
result = _op_def_lib.apply_op("Identity", input=input, name=name)
  File "/Users/kurt/anaconda2/envs/tensorflow/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 749, in apply_op
op_def=op_def)
  File "/Users/kurt/anaconda2/envs/tensorflow/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2380, in create_op
original_op=self._default_original_op, op_def=op_def)
  File "/Users/kurt/anaconda2/envs/tensorflow/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1298, in __init__
self._traceback = _extract_stack()

FailedPreconditionError (see above for traceback): Attempting to use uninitialized value pearson/pearson/covariance/count
 [[Node: pearson/pearson/covariance/count/read = Identity[T=DT_FLOAT, _class=["loc:@pearson/pearson/covariance/count"], _device="/job:localhost/replica:0/task:0/cpu:0"](pearson/pearson/covariance/count)]]

Sorry if the question is very basic - I'm quite a novice yet in TF. Could you please point out what's wrong here and how it can be fixed?

score 1 · Accepted Answer · edited May 23 '17 at 12:01

1

streaming_pearson_correlation function delegates to streaming_covariance, and it creates four local variables, according to documentation.

tf.initialize_all_variables() does not initialize local variables.

For Tensorflow r0.12+, use

tf.group(tf.global_variables_initializer(), tf.local_variables_initializer())

For Tensorflow <= r0.11, use

tf.group(tf.initialize_all_variables(), tf.initialize_variables(tf.local_variables()))

More: What is a local variable in tensorflow?

edited May 23 '17 at 12:01

Community

1
1

answered Jan 10 '17 at 16:32

standy

1,065
8
8

thanks @standby! I replaced 'sess.run(tf.initialize_all_variables())' with sess.run(tf.group(tf.global_variables_initializer(), tf.local_variables_initializer())) but I got this error: AttributeError: 'module' object has no attribute 'global_variables_initializer' I use TF 0.11.0rc0 – kurt Jan 10 '17 at 16:58
@standby In this post, it says I should use the previous one: http://stackoverflow.com/questions/40511562/tensorflow-module-object-has-no-attribute-global-variables-initializer Could you clarify on this? – kurt Jan 10 '17 at 17:06
@kurt initialize_all_variables was replaced with global_vairables_initializer in r0.12. I think you can use tf.initialize_variables(tf.local_variables()) in < r0.12 – standy Jan 10 '17 at 17:16
@standby that helped thanks but another similar problem turned up at another part of the network: tensorflow/core/framework/op_kernel.cc:968] Failed precondition: Attempting to use uninitialized value bwside1/BiRNN/FW/MultiRNNCell/Cell0/BasicLSTMCell/Linear/Matrix [[Node: bwside1/BiRNN/FW/MultiRNNCell/Cell0/BasicLSTMCell/Linear/Matrix/read = Identity[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](bwside1/BiRNN/FW/MultiRNNCell/Cell0/BasicLSTMCell/Linear/Matrix)]] Probably I should try to upgrade and see if it resolves this. – kurt Jan 10 '17 at 17:25
I mean you can use tf.initialize_variables(tf.local_variables()) instead of tf.local_variables_initializer(). You should still group it with tf.initialize_all_variables() to initialize global variables. – standy Jan 10 '17 at 17:30
@standby sorry I forgot to refresh the page so I didn't see your update. I went and updated my TF version and it solves the problem! cheers – kurt Jan 10 '17 at 17:46

Using Pearson Correlation instead of accuracy in tensorflow to report performance

1 Answers1