In machine learning and information theory, the cross entropy is a measure of distance (inverse similarity) between two probability distributions over the same underlying set of events. Cross entropy is the common choice of the loss function in neural networks for classification tasks.
Questions tagged [cross-entropy]
360 questions
1
vote
1 answer
Tensorflow R0.12 softmax_cross_entropy_with_logits ASSERT Error
I have been working on getting "softmax_cross_entropy_with_logits" working as part of my cost function for a 147 class problem. I have the code working with "sigmoid_cross_entropy_with_logits" but would like to move to softmax.
I have tried a…
mazecreator
- 543
- 1
- 11
- 27
1
vote
3 answers
Where is the origin coding of sparse_softmax_cross_entropy_with_logits function in tensorflow
I want to know what the tensorflow function sparse_softmax_cross_entropy_with_logits mathematically is exactly doing. But I can't find the origin of the coding.
Can you help me?
Alex
- 235
- 1
- 3
- 6
1
vote
1 answer
Does tf.nn.softmax_cross_entropy_with_logits account for batch size?
Does tf.nn.softmax_cross_entropy_with_logits account for batch size?
In my LSTM network, I feed batches of different sizes, and I would like to know whether or not I should normalize error with respect to batch size before optimizing.
casparjespersen
- 3,460
- 5
- 38
- 63
1
vote
1 answer
Torch CrossEntropyCriterion error
I'm trying to train a simple test network on the XOR function in Torch. It works when I use MSECriterion, but when I try CrossEntropyCriterion it fails with the following error message:
/home/a/torch/install/bin/luajit:…
rwallace
- 31,405
- 40
- 123
- 242
0
votes
0 answers
Find mathematical interpretation of loss function and back propagation
I have a code that uses machine learning and neural network. I used TensorFlow 2.0 and Keras. it is a classification program that gives output as 0 or 1. I used ReLU as activation function. Sparse SoftMax Cross Entropy is used as loss function.…
Rudra
- 1
0
votes
1 answer
Pytorch CE Loss evaluating to nan
I am training U-net with 3 channels, input data 40x40x40 set of voxels and using CELoss followed by ReLU activation. Also, there is only one convolution layer unlike the original U-net. For many iterations, my test loss evaluates to nan but train…
Pranjali Singh
- 1
- 1
0
votes
1 answer
why does my neural network coded from scratch results have such a weird loss trend?
I am developing neural network from scratch. It consists of of the following input > layer1(sigmoid) > layer2 > output(softmax). The basic coding is complete but when I ran it, I obtain a weird loss plot. Also if I ran the code for large iterations…
gingerorange
- 9
- 3
0
votes
1 answer
Regression with numerical labels Y={1,2,3} VS Classification with labels Y={[1 0 0],[0 1 0],[0 0 1]}?
Assume a deep learning problem, where there exists only one object in the image, we want to classify whether the object is either
Y={Cat:1, Dog:2, Panda:3}
Can we address this problem using neural networks in two ways:
Regression Approach:…
Abdallah Ahmed
- 47
- 4
0
votes
0 answers
How do I use Cross Entropy to evaluate my model
Can i get a CE score just like F1 score? I have been searching for a concise solution I am new to this? I saw this paper where Cross entropy was computed as the evaluation metric but it not clear how to implement this. I have searched hugging face…
smart_me
- 1
- 2
0
votes
1 answer
PyTorch torch.nn.Functional - Cross Entropy Loss Multi-class Classification with Integer Labels
Is there a method in "torch.nn.Functional" which can allow me to get the Cross Entropy Loss for a Multi-class Classification with Integer Labels (1 integer label/class per instance)?
Should I convert both lists into a FloatTensor?
import…
Kai
- 17
- 7
0
votes
0 answers
PyTorch RuntimeError: Expected floating point type for target with class probabilities, got Long
I am trying to calculate the validation loss for my object detection model using Pytorch. My pre-trained model is faster rcnn. Below is the function I used to calculate the validation loss
import torch.nn.functional as F
def…
user21705974
- 1
- 1
0
votes
1 answer
CrossEntropyLoss using weights gives RuntimeError: expected scalar type Float but found Long neural network
I am using a Feedforward neural network for a classification task with 4 classes. The classes are imbalanced and hence, I want to use a weight with the CrossEntropyLoss as mentioned here.
Here is my neural network:
class…
diviquery
- 569
- 5
- 19
0
votes
0 answers
Validation loss for sparse_categorical_crossentropy is coming up as nan consistently
In training (sparse_categorical_crossentropy ) loss is coming up with a loss value but the validation (sparse_categorical_crossentropy) gives back nan.
It is a multi output model. One is a binary_crossentropy (which results a loss) and one is…
CrumpledMemories
- 246
- 1
- 8
0
votes
0 answers
LightGBM Probabilities calibration with custom cross-entropy score and loss function
I am currently trying to perform LightGBM probabilities calibration with a custom cross-entropy score and loss function for a binary classification problem. My issue is related to the custom cross-entropy that leads to incompatibility with…
Sosa
- 141
- 9
0
votes
1 answer
Pytorch nn.CrossEntropyLoss() only returns -0.0
Running the following code snippet
torch.nn.CrossEntropyLoss()(torch.Tensor([0]), torch.Tensor([1]))
returns
tensor(-0.)
How can this be? Am I missing something fundamental about this problem?
I have a super simple Feed-Forward NN model:
import…
Selvi7
- 1