In machine learning and information theory, the cross entropy is a measure of distance (inverse similarity) between two probability distributions over the same underlying set of events. Cross entropy is the common choice of the loss function in neural networks for classification tasks.
Questions tagged [cross-entropy]
360 questions
4
votes
1 answer
How to correctly use Cross Entropy Loss vs Softmax for classification?
I want to train a multi class classifier using Pytorch.
Following the official Pytorch doc shows how to use a nn.CrossEntropyLoss() after a last layer of type nn.Linear(84, 10).
However, I remember this is what Softmax does.
This leaves me…
Gulzar
- 23,452
- 27
- 113
- 201
4
votes
3 answers
Weighted sparse categorical cross entropy
I am dealing with a semantic segmentation problem where the two classes in which I am interested (in addition to background) are quiet unbalanced in the image pixels. I am actually using sparse categorical cross entropy as a loss, due to the way in…
Vitto
- 361
- 3
- 17
4
votes
1 answer
How to adjust for the grading bias of labels in a classification task?
I am currently working on a convolutional neural network for pathological changes detection on x-ray images. It is a simple binary classification task. In the beginning of the project we gathered around 6000 x-rays and asked 3 different doctors…
Maju116
- 1,607
- 1
- 15
- 30
4
votes
1 answer
Apply a PyTorch CrossEntropy method for multiclass segmentation
I am trying to implement a simple example of how to apply cross-entropy to what is supposed to be the output of my semantic segmentation CNN.
Using the pytorch format I would have something like this:
out = np.array([[
[
[1.,1, 1],
…
lvl
- 109
- 3
- 6
4
votes
1 answer
Keras - what accuracy metric should be used along with sparse_categorical_crossentropy to compile model
When I have 2 classes I used binary_crossentropy as loss value like this to compile a model:
model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['accuracy'])
But right now I have 5 classes & I'm not using on hot encoded…
Poles
- 3,585
- 9
- 43
- 91
4
votes
1 answer
Pytorch LSTM: Target Dimension in Calculating Cross Entropy Loss
I've been trying to get an LSTM (LSTM followed by a linear layer in a custom model), working in Pytorch, but was getting the following error when calculating the loss:
Assertion cur_target >= 0 && cur_target < n_classes' failed.
I defined the loss…
LunarLlama
- 229
- 2
- 11
4
votes
2 answers
why softmax_cross_entropy_with_logits_v2 return cost even same value
i have tested "softmax_cross_entropy_with_logits_v2"
with a random number
import tensorflow as tf
x = tf.placeholder(tf.float32,shape=[None,5])
y = tf.placeholder(tf.float32,shape=[None,5])
softmax =…
UfXpri
- 53
- 1
- 4
4
votes
2 answers
Channel wise CrossEntropyLoss for image segmentation in pytorch
I am doing an image segmentation task. There are 7 classes in total so the final outout is a tensor like [batch, 7, height, width] which is a softmax output. Now intuitively I wanted to use CrossEntropy loss but the pytorch implementation doesn't…
Farshid Rayhan
- 1,134
- 4
- 17
- 31
4
votes
4 answers
Comparing MSE loss and cross-entropy loss in terms of convergence
For a very simple classification problem where I have a target vector [0,0,0,....0] and a prediction vector [0,0.1,0.2,....1] would cross-entropy loss converge better/faster or would MSE loss?
When I plot them it seems to me that MSE loss has a…
aa1
- 783
- 1
- 15
- 31
4
votes
1 answer
Tensorflow: Weighted sparse softmax with cross entropy loss
I am doing image segmentation using fully convolutional neural networks (link to the paper): https://people.eecs.berkeley.edu/~jonlong/long_shelhamer_fcn.pdf
This can be considered as pixel classification (in the end each pixel is getting a label)
I…
TheRevanchist
- 331
- 1
- 4
- 12
4
votes
1 answer
cross entropy is nan
I am deploying my conv-deconv net. My question is the cross entropy was always nan while training so the solver didn't update the weights. I checked my code all day but I didn't know where did I go wrong. The following is my architecture:
here is…
clickListener
- 69
- 1
- 1
- 5
4
votes
1 answer
Tensorflow: scaled logits with cross entropy
In Tensorflow, I have a classifier network and unbalanced training classes. For various reasons I cannot use resampling to compensate for the unbalanced data. Therefore I am forced to compensate for the misbalance by other means, specifically…
Ron Cohen
- 2,815
- 5
- 30
- 45
3
votes
0 answers
"RuntimeError: Expected target size" error for the nn.CrossEntropyLoss() function
I am trying to train a GPT-2 model to take in a tokenized/padded input and predict the output. My batch size is 32. My max length is 343. I believe that the 768 comes from the model. I cannot get the loss function to work properly though. The…
C_Dog
- 31
- 2
3
votes
1 answer
Cross Entropy for Soft Labeling in Pytorch
i'm trying to define the loss function of a two-class classification problem. However, the target label is not hard label 0,1, but a float number between 0~1.
torch.nn.CrossEntropy in Pytorch do not support soft label so i'm trying to write a cross…
Jinting Zhao
- 89
- 4
3
votes
1 answer
Correct use of Cross-entropy as a loss function for sequence of elements
I have a sequece labeling task.
So as input, I have a sequence of elements with shape [batch_size, sequence_length] and where each element of this sequence should be assigned with some class.
And as a loss function during training a neural net, I…
Kenenbek Arzymatov
- 8,439
- 19
- 58
- 109