In machine learning and information theory, the cross entropy is a measure of distance (inverse similarity) between two probability distributions over the same underlying set of events. Cross entropy is the common choice of the loss function in neural networks for classification tasks.
Questions tagged [cross-entropy]
360 questions
0
votes
0 answers
How should we process if someone wishes to increase/maximize the categorical cross-entropy loss for a certain scenario?
In one of my projects, I need to maximize the entropy loss in a classification model. How should I write the codebase? I'm compatible with keras, tf and pytorch framework.
Any kind of suggestions and direction will be appreciated.
Thanks in advance.
Md. Tariqul Islam
- 23
- 1
- 7
0
votes
1 answer
focal loss for imbalanced data using pytorch
I want to use focal loss with multiclass imbalanced data using pytorch . I searched got and try to use this code but I got error
class_weights=tf.constant([0.21, 0.45, 0.4, 0.46, 0.48,…
Shorouk Adel
- 127
- 3
- 20
0
votes
1 answer
Cross-entropy validation losses comes out as a straight line
I'm trying to calculate cross-entropy losses using the Iris dataset, but when I ran my model and fired up my plots, both my losses and validation losses remained a straight line at zero. I don't know what I'm doing wrong. This is my code:
import…
javanewb1
- 33
- 5
0
votes
1 answer
How to implement a modified cross entropy loss function?
I am currently working on a change detection project for my university course and I was stuck at writing a custom loss function.I know i have to use function closure to be able to use data from layers of the model but i don't know enough…
akshithbellare
- 1
- 1
0
votes
1 answer
I am trying to build custom loss function in pytorch, but I am getting nan loss and accuracy is not improving
class crossentropy(nn.Module):
def __init__(self):
super(crossentropy, self).__init__()
def forward(self, y_1, y):
m = nn.Softmax(dim=1)
output = m(y_1)
loss = -1.0*torch.sum(y*torch.log(output))
l =…
abhilash sharma
- 11
- 2
0
votes
1 answer
1. Weighted Loss in CrossEntropyLoss() 2. Combination of WeightedRandomSampler and subsampler
I wanted to implement class weights to my 3 class classification problem.
Tried by just directly adding the weights, which gives me an error when passing my model output and the labels to my loss
criterion =…
patrick823
- 131
- 5
0
votes
1 answer
IndexError: Target 11 is out of bounds. cross-entropy
How can I change the attached model to fit my dataset for the Bayesian model? my data include 5 variables and 32 results
model = nn.Sequential(
bnn.BayesLinear(prior_mu=0, prior_sigma=0.1, in_features=5, out_features=100),
nn.ReLU(),
…
Yumeng Xu
- 179
- 1
- 2
- 11
0
votes
1 answer
Why is the binary cross entropy loss during training of tf model different than that calculated by sklearn?
I am building a neural collaborative filtering recommendation model using tensorflow, using binary cross entropy as the loss function. The labels to be predicted are, of course, binary.
Upon training each epoch, the loss function is printed. I have…
Jeremy
- 67
- 4
0
votes
0 answers
What am I doing incorrectly here that does leaves my accuracy as 0% in my model, even though I know my model works well on the iris data set?
model = nnet(4, 2, 1)
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.SGD(model.parameters(), lr=0.1)
trainloader = Dataloader(dataset=dataset, batch_size = 15)
val_loader = Dataloader(dataset=val_dataset, batch_size = 150)
LOSS =…
Robbie Meaney
- 21
- 2
0
votes
1 answer
tf.keras.losses.CategoricalCrossentropy gives different values than plain implementation
Any one knows why raw implementation of Categorical Crossentropy function is so different from the tf.keras's api function?
import tensorflow as tf
import math
tf.enable_eager_execution()
y_true =np.array( [[1., 0., 0.], [0., 1., 0.], [0., 0.,…
Jason
- 3,166
- 3
- 20
- 37
0
votes
0 answers
How to use nn.CrossEntropyLoss() for a PatchGAN Discriminator output?
I am trying to use the nn.CrossEntropyLoss() to find the cross-entropy loss between reals and fakes of a patchGAN discriminator that outputs a tensor of shape (batch_size, 1, 30, 30).
I am confused with the documentation here that asks for class…
amalp12
- 185
- 2
- 10
0
votes
1 answer
Proper way to use Cross entropy loss with one hot vector in Pytorch
I've encountered many suggestions by searching online, but I don't understand the proper way to do it.
Lets say my output of the model is 4 neurons, and the target labels are 1000 0100 0010 0001.
In tensorflow, I added a softmax layer at the end,…
Adar Cohen
- 158
- 1
- 13
0
votes
0 answers
How get and print every class's weight in PyTorch cross entropy loss function
I have code for multiclass segmentation using PyTorch. The inputs are images and their ground truth masks. This a piece of my code:
criterion = nn.CrossEntropyLoss()
for epoch in range(epochs):
net.train()
epoch_loss = 0
for batch in…
Babak Azad
- 60
- 1
- 7
0
votes
1 answer
pytorch cross-entropy-loss weights not working
I was playing around with some code and and it behaved differently than what i expected. So i dumbed it down to a minimally working example:
import torch
test_act = torch.tensor([[2.,0.]])
test_target = torch.tensor([0])
loss_function_test =…
Eumel
- 1,298
- 1
- 9
- 19
0
votes
1 answer
UNET with CrossEntropy Loss Function
I was trying to train UNET with input size as [3,128,128] and the corresponding mask is [1,128,128] which contains classes directly(instead of pixels it will contain class numbers - 1,2). I am trying for a two-class problem hence my mask contains…
Rajesh
- 221
- 4
- 16