0
import numpy as np
fake_preds = np.random.rand(10,768)  # assuming predictions of some model (10 examples) 

fake_labels = np.random.rand(10,1)   # fake labels for 10 examples

loss1 = tf.keras.losses.BinaryCrossentropy(from_logits=True)

print(loss1(fake_labels, fake_preds).numpy()) 

--> Throws shape mismatch error:

ValueError: `logits` and `labels` must have the same shape, received ((10, 768) vs (10, 1)).

loss2 = tf.keras.losses.BinaryCrossentopy(from_logits=False)

print(loss2(fake_labels, fake_preds),numpy())

--> 0.998 (some value for random labels and preds), works well!

Why???

Keras Documentation statest the shape of logits and targets must be same for loss functions to work properly. Then why is it that there is no shape mismatch error on from_logits = False!

D.L
  • 4,339
  • 5
  • 22
  • 45
  • This looks like a bug, you can open an issue on their Github. `from_logits=True` calls `sigmoid_cross_entropy_with_logits` and it has shape validation. Otherwise Keras calculates it and applies broadcasting(?) due to shape being (10,1) while multiplying. https://github.com/keras-team/keras/blob/e6784e4302c7b8cd116b74a784f4b78d60e83c26/keras/backend.py#L5687-L5690 – Frightera Jan 01 '23 at 00:26
  • Thanks for the link @Frightera, I opened up an issue – Rahul Ohlan Jan 02 '23 at 01:31

0 Answers0