Does following code give recall for multiclass classification in Keras?

Question

Does following code give recall for multiclass classification in Keras? even though I am not passing y_true and y_pred while calling recall function in model.compile, it showed me result of the recall.

def recall(y_true, y_pred):
    y_true = K.ones_like(y_true) 
    true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))
    all_positives = K.sum(K.round(K.clip(y_true, 0, 1)))
    
    recall = true_positives / (all_positives + K.epsilon())
    return recall

model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=[recall])

score 1 · Answer 1 · answered Aug 28 '22 at 06:47

Yes, it works because recall is called multiple times under the hood inside model.fit() specifying those values.

It works in a way similar (more complex and optimized) to this:

accuracy = tf.keras.metrics.CategoricalAccuracy()
loss_fn = tf.keras.losses.CategoricalCrossentropy(from_logits=True)
optimizer = tf.keras.optimizers.Adam()

# Iterate over the batches of a dataset.
for step, (x, y) in enumerate(dataset):
    with tf.GradientTape() as tape:
        logits = model(x)
        # Compute the loss value for this batch.
        loss_value = loss_fn(y, logits)

    # Update the state of the `accuracy` metric.
    accuracy.update_state(y, logits)

    # Update the weights of the model to minimize the loss value.
    gradients = tape.gradient(loss_value, model.trainable_weights)
    optimizer.apply_gradients(zip(gradients, model.trainable_weights))

    # Logging the current accuracy value so far.
    if step % 100 == 0:
        print('Step:', step)        
        print('Total running accuracy so far: %.3f' % accuracy.result())

This is called a Gradient Tape, and it can be used to perform a customized train loop. Basically it exposes the gradients computed on the trainable tensors of your model. It lets you update the weights of the model manually, so it is really useful for personalization. All this stuff is also done automatically inside model.fit(). You don't need this, it is just to explain how things work.

As you can see, at every batch of the dataset are computed the predictions, that is, the logits. The logits and the ground truth, that is the correct y values, are given as arguments to accuracy.update_state, just as it is done without you seeing it inside model.fit(). Even the order is the same, y_true and y are both the ground truth, and y_pred and logits the predictions.

I hope this has made things clearer.

Thank you for your explanation. By using the code, I mentioned, I got recall 100% and accuracy 75% for multiclass classification. How that could be possible for recall? Can you explain what is the role of y_true * y_pred in true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))? Thank you in advance. — Sheetal Nagar, Aug 29 '22 at 07:13
I'll explain why `y_true*y_pred`. Remember that you are working with tensors and in a metric function you can't use normal python stuff to count all correct predictions. Your y_true is probably a label one-hot encoded e.g. [0 0 1] that is that sample belongs to the third class. y_pred are probably logits or a probability, e.g. [0 0.2 0.8]. The model is 80% confident that the sample belongs to the third class. You want to get the number of true positives, one way to do this is to multiply [0 0 1]*[0 0.2 0.8]=[0 0 0.8]. That 0.8 rounded will get to 1 and counts as a correct prediction. — ClaudiaR, Aug 29 '22 at 17:34
If you had [0 0 1]*[0.8 0.2 0]=[0 0 0], then you have an incorrect prediction, that adds nothing to the final K.sum where all your values are summed to count the number of correct predictions. Hope things are clearer now. — ClaudiaR, Aug 29 '22 at 17:35
Okay. I got it. but isn't it wrong to multiply `y_true*y_pred` to find correct recall value. — Sheetal Nagar, Dec 03 '22 at 11:16
@SheetalNagar I think the same. When you roughly do `K.round(y_true*y_pred)` then you will end up with binary classification because only predictions equal or bigger than 0.5 will be taken as correct. — Gustaw Ohler, Jul 04 '23 at 13:10

Does following code give recall for multiclass classification in Keras?

1 Answers1