Different cross entropy results from NumPy and PyTorch

Question

My prediction is y_hat = [ 0.57,0.05,0.14,0.10,0.14] and target is target =[ 1, 0, 0, 0, 0 ].

I need to calculate Cross Entropy loss by NumPy and Pytorch loss function.

Using NumPy my formula is -np.sum(target*np.log(y_hat)), and I got 0.5621189181535413

However, using Pytorch:

loss = nn.CrossEntropyLoss()
output = torch.FloatTensor([0.57,0.05,0.14,0.10,0.14])
label = torch.FloatTensor([1,0,0,0,0])
loss_value = loss(output, label)
print(loss_value)

Gives tensor(1.2586), which is different.

Please don't completely change the wording and meaning of a question, especially after you've gotten an answer. If you have a new question, please ask it as such. — Adriaan, Oct 21 '22 at 12:05

score 1 · Answer 1 · edited Oct 18 '22 at 17:51

You need to apply the softmax function to your y_hat vector before computing cross-entropy loss. For example, you can use scipy.special.softmax().

>>> from scipy.special import softmax
>>> import numpy as np
>>> y_hat = [0.57, 0.05, 0.14, 0.10, 0.14]
>>> target =[1, 0, 0, 0, 0]
>>> y_hat = softmax(y_hat)
>>> -np.sum(target * np.log(y_hat))
1.2586146726011722

Which agrees with the result from Pytorch.

Different cross entropy results from NumPy and PyTorch

1 Answers1