In machine learning and information theory, the cross entropy is a measure of distance (inverse similarity) between two probability distributions over the same underlying set of events. Cross entropy is the common choice of the loss function in neural networks for classification tasks.
Questions tagged [cross-entropy]
360 questions
0
votes
1 answer
'RuntimeError: Expected object of scalar type Long but got scalar' for torch.nn.CrossEntropyLoss()
I'm using this loss function for xlm-roberta-large-longformer and it gives me this error:
import torch.nn.functional as f
from scipy.special import softmax
loss_func = torch.nn.CrossEntropyLoss()
output =…
Niloufar Modir
- 31
- 4
0
votes
1 answer
What is "cross entropy loss" really doing when input is 3D?
I'm working on a text-generating RNN. I found out that when calculating cross entropy loss, if the input has a size of[batch_size, vocab_size, seq_len] and the target has a size of[batch_size, seq_len], then the model is not working at all however…
Na1ve
- 11
- 3
0
votes
1 answer
VGG-16 and ResNet-9 loss values not corresponding to accuracy on test set
I have two models whose performance I am comparing, a ResNet-9 model and a VGG-16 model. They are being used for image classification. Their accuracies on the same test set are:
ResNet-9 = 99.25% accuracy
VGG-16 = 97.90% accuracy
However from…
Slwd-wave540
- 59
- 5
0
votes
1 answer
Why my cross entropy loss function does not converge?
I try to write a cross entropy loss function by myself. My loss function gives the same loss value as the official one, but when i use my loss function in the code instead of official cross entropy loss function, the code does not converge. When i…
zhuzhu
- 13
- 2
0
votes
1 answer
Cannot verify manually the calculations by tf.keras.losses.BinaryCrossentropy
https://www.tensorflow.org/api_docs/python/tf/keras/losses/BinaryCrossentropy
The example in tensorflow site.
y_true = [0, 1, 0, 0]
y_pred = [-18.6, 0.51, 2.94, -12.8]
bce = tf.keras.losses.BinaryCrossentropy(from_logits=True)
bce(y_true,…
string
- 39
- 4
0
votes
1 answer
Why the result of categorical cross entropy in tensorflow different from the definition?
I am testing outcomes of tf.keras.losses.CategoricalCrossEntropy, and it gives me values different from the definition.
My understanding of cross entropy is:
def ce_loss_def(y_true, y_pred):
return tf.reduce_sum(-tf.math.multiply(y_true,…
EulersRealPie
- 26
- 1
- 4
0
votes
0 answers
Pytorch - F.cross_entropy loss for sequence classification - Correct dimensions?
I am trying to perform sequence classification using a custom implementation of a transformer encoder layer. I have been following this tutorial pretty faithfully:
tutorial.
The tutorial, however, does not demonstrate an example of using this model…
Desperados
- 434
- 5
- 13
0
votes
1 answer
Accuracy 0% for binary classification
I am using the OpenFL framework for doing Federated Learning experiments. I run their tutorial notebooks without problems, so for example I am able to run classification on MNIST and everything is ok.
Now I am using 2 clients with 2 different…
CasellaJr
- 378
- 2
- 11
- 26
0
votes
1 answer
How can I separate the the binary cross entropy loss with and add weight for different class/label?
I am trying custom the binary cross entropy loss from the paper by Pytroch, but I meet some problems here. I am not sure how can I use Pytorch here to get the target y when I let predict label (sigmoid(input)) == 1. Can someone help me with that? Or…
Bertie
- 31
- 2
0
votes
1 answer
Is it possible to use PyTorch's `BatchNorm1d` with `BCELossWithLogits`?
I am attempting to normalize the outputs of my classifier that uses BCELossWithLogits as part of its loss function. As far as I know, this implements Sigmoid function internally and outputs the loss.
I want to normalize the output of the sigmoid…
c21253
- 330
- 4
- 13
0
votes
0 answers
Activation function in segmentation network and metrics computation
I had a binary segmentation task: I had to predict yes or no for each pixel of an image.
Therefore I used a binary cross entropy loss (which is defined in Pytorch and combines a sigmoid and a cross entropy loss) to train the network.
To compute the…
emanuele_f
- 35
- 1
- 5
0
votes
0 answers
neural network binary classification softmax logsofmax and loss function
I am building a binary classification where the class I want to predict is present only <2% of times. I am using pytorch
The last layer could be logosftmax or softmax.
self.softmax = nn.Softmax(dim=1) or self.softmax = nn.LogSoftmax(dim=1)
my…
user2543622
- 5,760
- 25
- 91
- 159
0
votes
1 answer
binary_crossentrophy vs categorical_crossentropy
I have a dataset with 10 categorical features and one output feature with class 0 and 1. X_train follows a 3D array so I have done label encoding beforehand on the dataset.
I have applied categorical_crossentrophy but I am getting 26% accuracy with…
be_real
- 180
- 1
- 1
- 12
0
votes
0 answers
Error in torch.nn.functional.cross_entropy function in PyTorch
I implemented a code and I am trying to compute torch.nn.functional.cross_entropy but unfortunately, I receive the RuntimeError: only batches of spatial targets supported (3D tensors) but got targets of size: : [256] error!
cuda =…
Kasra.F
- 1
0
votes
0 answers
Cross Entropy loss function returns 0.001 on image segmentation neural network predictions, but no prediction has sense?
I'm creating a neural network in pytorch that takes an 250x260 image in 3 channels as an input, and returns a pixel by pixel classification (4 classes), it returns 4 channels of 250x260 images representing each class.
This is the neural…
Agustin Ferrante
- 13
- 4