3

I'm running tf2.0 and simply can not print the confusion matrix values. The problem is described below.

  @tf.function
  def test_step(self, x , y):
    predictions = model(x, training=False)
    loss = self.loss(y, predictions)

    y, predictions = tf.reshape(y,[-1,]), tf.reshape(predictions, [-1,])

    # Cast into class labels
    predictions = math_ops.cast(predictions > 0.5, predictions.dtype)

    ....

    self.test_conf_matrix = tf.math.confusion_matrix(y, predictions, num_classes=2) <--- important line!

Everything is going well so far, and the confusion matrix will be computed properly.

But it is simply not possible to print it out in the end like:

print(str(self.test_conf_matrix.numpy()))

The error I get is:

AttributeError: 'Tensor' object has no attribute 'numpy'

But since tf2 and eagerExecution this should be done this way, right? See: TF2.0 Tutorial

SamSampleman
  • 65
  • 2
  • 8
  • What's `math_ops`. Where did you import that from? – thushv89 Jan 19 '20 at 21:56
  • It is from 'from tensorflow.python.ops import math_ops'. Because e.g. the F1-Score implementation of tfa does not work with probabilities. So I had a look at tensorflows accuracy implementation and that is the way they cast the probabilities into class labels. – SamSampleman Jan 20 '20 at 15:15

2 Answers2

2

According to the definition of tf.function,

"Compiles a function into a callable TensorFlow graph".

Since the tf.function imposes a TensorFlow graph, you cannot use anything outside of the tf.* methods.

That means that any arbitrary python code cannot be used inside a tf.function, only what is already available in the tf.* methods.

The same exact phenomenon happens when you want to iterate on a tf.data.Dataset with a map function. That map function you want to use on tf.data.Dataset cannot contain arbitrary python code, unless you specifically use a tf.py_function.

These operation are specifically executed in graph mode for performance reasons, and thus, you cannot call methods that belong to the 'eager execution' category, such as .numpy().

Timbus Calin
  • 13,809
  • 5
  • 41
  • 59
  • So, thanks but I don't really get it. The documentation of tf.math.confusion_matrix says it returns a Tensor, and that is what it does. So when I have a look at the doc of a Tensor there is the .numpy() function described. Do you have a suggestion how to overcome the issue? – SamSampleman Jan 20 '20 at 15:20
  • You cannot. You just cannot use .numpy() inside a function that is used in a TensorFlow graph. – Timbus Calin Jan 20 '20 at 16:10
  • 2
    The only way you can use .numpy() or so is to call/create a specific py_fuunction. The fact that .numpy() exists in the documentation for a tensor does not mean you can apply it everywhere :D – Timbus Calin Jan 20 '20 at 16:11
  • Haben Sie verstanden? – Timbus Calin Jan 21 '20 at 07:02
  • No, not completely. Because I can process any other evaluation metrics like tf.keras.metrics.BinaryAccuracy. So do you have a solution how to get the result from the confusion matrix? I don't get the difference between one of those evaluation metrics and the confusion matrix. Why can I print / evaluate all other metrics but the confusion matrix? – SamSampleman Jan 21 '20 at 15:03
  • Because tf.*(like keras and tf.math.*) is part of the TensorFlow functions, that's why they can be used. – Timbus Calin Jan 22 '20 at 11:53
  • So, tf.math.confusion_matrix() is part of the TF functions...But It can't be used. – SamSampleman Jan 22 '20 at 20:57
  • It can be used. But you cannot convert it to numpy(). The .numpy() problem on the result of tf.* functions in your case is the problem. Whatever tf.* method you use(suppose that this is a method you apply on dataset), you cannot convert it to .numpy(). That is the issue, when you try to convert a TensorFlow tensor to numpy in your case. You cannot; If the other metrics that you say work, that is because they are optimized behind the scenes with TensorFlow tensors.... but .numpy() is out of tf.* library. Do you understand now? I don't think I can explain better than this. – Timbus Calin Jan 22 '20 at 21:44
  • Please consult this question to have a better overview if it's still blurry: https://stackoverflow.com/questions/59497372/is-there-an-alternative-to-tf-py-function-for-custom-python-code – Timbus Calin Jan 22 '20 at 21:47
  • So, what's the solution, Timbius? –  Feb 18 '22 at 17:31
0

Look at : https://www.tensorflow.org/api_docs/python/tf/numpy_function

def my_numpy_func(x):
  # x will be a numpy array with the contents of the input to the
  # tf.function
  return np.sinh(x)

@tf.function(input_signature=[tf.TensorSpec(None, tf.float32)])
def tf_function(input):
  y = tf.numpy_function(my_numpy_func, [input], tf.float32)
  return y * y
tf_function(tf.constant(1.))

This works for me fine.

You make a function something like

def print_cm(cm):
  print(cm)

@tf.function()
 def test_step(self, x , y):
   
    ....

    self.test_conf_matrix = tf.math.confusion_matrix(y, predictions, num_classes=2) # <--- important line!
    # make print as numpy array under tf.function decorator 
    print_cm(test_conf_matrix))  # just call function
megamachine
  • 1
  • 1