In machine learning and information theory, the cross entropy is a measure of distance (inverse similarity) between two probability distributions over the same underlying set of events. Cross entropy is the common choice of the loss function in neural networks for classification tasks.
Questions tagged [cross-entropy]
360 questions
1
vote
0 answers
Unexpected behaviour on using class weights in loss
I’m working on a classification problem (500 classes). My NN has 3 fully connected layers, followed by an LSTM layer. I use nn.CrossEntropyLoss() as my loss function. To tackle the problem of class imbalance, I use sklearn’s class_weight while…
theodre7
- 125
- 4
1
vote
1 answer
difference in categorical crossentropy when specified in loss or metrics
What is the difference between
tf.keras.losses.CategoricalCrossentropy and tf.keras.metrics.CategoricalCrossentropy?
model.compile(optimizer=Adam(learning_rate=lr), loss= 'categorical_crossentropy',…
Seena
- 101
- 1
- 8
1
vote
1 answer
Why is the Tensorflow and Pytorch CrossEntropy loss returns different values for same example
I have tried getting Tensorflow and Pytorch CrossEntropyLoss but it returns different values and I don't know why. Please find the below code and results. Thanks for your inputs and help.
import tensorflow as tf
import numpy as np
y_true = [3, 3,…
ankur singhania
- 51
- 6
1
vote
0 answers
InvalidArgumentError: Graph execution error: Node: 'categorical_crossentropy/remove_squeezable_dimensions/Squeeze'
I want to train U-Net for a semantic segmentation task using cross entropy as a loss function. I have RGB images and masks. The mask is one hot encoded. When I try to train the model, I got this error.
seed=123
batch_size= 32
n_classes=2
from…
marlowebe
- 11
- 2
1
vote
0 answers
Increasing the cross entropy loss instead of decreasing it
Is there a way to flip the effect of the cross-entropy loss?
I have a language model, and I want to train the model in a way that doesn't generate a specific text. Thus, I have two losses, one that I want to reduce (loss1) and another that I want to…
Minions
- 5,104
- 5
- 50
- 91
1
vote
0 answers
Difference between binary cross entropy with sigmoid cross entropy
What is difference between "Binary cross entropy" with "Sigmoid cross entropy"?
ttaaee98
- 35
- 3
1
vote
2 answers
Do I need to apply the Softmax Function ANYWHERE in my multi-class classification Model?
I am currently turning my Binary Classification Model to a multi-class classification Model. Bare with me.. I am very knew to pytorch and Machine Learning.
Most of what I state here, I know from the following…
patrick823
- 131
- 5
1
vote
2 answers
using nn.Cross entropy between outputs and target label
I use this code
function to train the model
def train():
model.train()
total_loss, total_accuracy = 0, 0
# empty list to save model predictions
total_preds=[]
# iterate over batches
for step,batch in…
Shorouk Adel
- 127
- 3
- 20
1
vote
2 answers
manually computing cross entropy loss in pytorch
I am trying to compute cross_entropy loss manually in Pytorch for an encoder-decoder model.
I used the code posted here to compute it: Cross Entropy in PyTorch
I updated the code to discard padded tokens (-100). The final code is this:
class…
Minions
- 5,104
- 5
- 50
- 91
1
vote
1 answer
Calculation cross entropy for batch of two tensors
I’d like to calculate cross entropy for batch of two tensors:
x = torch.tensor([[[ 2.1137, -1.3133, 0.7930, 0.3330, 0.9407],
[-0.8380, -2.0299, -1.1218, 0.3150, 0.4797],
[-0.7439, 0.0753, -0.1121, 0.0096, -1.2621]]])
y =…
Roman Kazmin
- 931
- 6
- 18
1
vote
1 answer
Numpy array indexing with complete vector
What does the following code do? (grad[range(m),y] -= 1)
def delta_cross_entropy(X,y):
"""
X is the output from fully connected layer (num_examples x num_classes)
y is labels (num_examples x 1)
Note that y is not one-hot encoded…
Clebo Sevic
- 581
- 1
- 7
- 17
1
vote
1 answer
Getting RESNet18 to work with float32 data
I have float32 data that I am trying to get RESNet18 to work with. I am using the RESNet model in torchvision (and using pytorch lightning) and modified it to use one layer (grayscale) data like so:
class ResNetMSTAR(pl.LightningModule):
def…
DLH
- 199
- 11
1
vote
1 answer
pytorch custom loss function nn.CrossEntropyLoss
After studying autograd, I tried to make loss function myself.
And here are my loss
def myCEE(outputs,targets):
exp=torch.exp(outputs)
A=torch.log(torch.sum(exp,dim=1))
hadamard=F.one_hot(targets, num_classes=10).float()*outputs
…
Jake
- 65
- 4
1
vote
1 answer
Pytorch LSTM and cross entropy
I am working on sentiment analysis, I want to classify the output into 4 classes. For loss I am using cross-entropy.
The problem is PyTorch cross-entropy needs the input of (batch_size, output) which is am having trouble with.
I am taking a batch…
arun kothari
- 13
- 2
1
vote
1 answer
Configuring labels in TensorFlow BinaryCrossentropy loss function
I want to compute cross-entropy loss using tf.keras.losses.BinaryCrossentropy. The documentation has the following example, and specifies that true labels and predicted labels should have the shape [batch_size]:
y_true = [[0., 1.], [0., 0.]]
y_pred…
Reveille
- 4,359
- 3
- 23
- 46