Stack Overflow
Stack Overflow

Questions tagged [cross-entropy]

In machine learning and information theory, the cross entropy is a measure of distance (inverse similarity) between two probability distributions over the same underlying set of events. Cross entropy is the common choice of the loss function in neural networks for classification tasks.

360 questions
0
votes
0 answers

How calculate derivative of cross entropy loss function?

Let's say I have a vector y_true, and a vector y_pred. I calculate the cross entropy between the two vectors, but for the gradient backpropagation, I don't know how to calculate the derivative of the cross entropy. How to calculate the derivative of…
Nicolas Brugie
  • 35
  • 6
0
votes
1 answer

How can I minimize parameters of a Weibull Distribution using Kullback-Leibler method in Python?

I want to find the parameters of a Weibull distribution by minimizing the parameters using Kullbak-Leibler method. I found a code here which did the same thing. I replaced the Normal distributions in the original code by the Weibull distributions. I…
NeSha
  • 1
  • 1
0
votes
0 answers

Can cross entropy loss be used to compare how well a model performs on different parts of a dataset?

Say that I have a dataset with binary labels and a binary classification model. I then split the dataset into N parts. Now if I measure the cross-entropy loss on each split, will the resulting cross-entropies become objective measures of how well…
Ferus
  • 1,080
  • 3
  • 12
  • 17
0
votes
1 answer

Cross entropy IndexError Dimension out of range

I'm trying to train a GAN in some images, I followed the tutorial on pytorch's page and got to the following code, but when the crossentropy function is applyed during the training it returns the error below the code: import random import torch.nn…
Ramon Griffo
  • 333
  • 5
  • 14
0
votes
1 answer

How to handle Variable Sized inputs at the end of CNN layers before sending to the Loss Function

I am dealing with variable sized inputs to a CNN and I wanted to know how to feed it to a last FC layer to satisfy the requirement for the CrossEntropy Loss function. Even if taken care for one sample, the subsequent sample would have different…
Dheeraj P R
  • 13
  • 2
0
votes
1 answer

How can I convert the dimension in the model form 2D to 1D?

I am beginner of using pytorch. I would like to classify 2d binary array (17 * 20 ) to 8 classes, I am using cross entropy as loss function . I have 512 batch size . the input is 512 batches of size (17 * 20 )and the final outpu 512 batches of size…
No Na
  • 27
  • 2
  • 7
0
votes
1 answer

Not the same result for log loss and cross entropy loss

The negative log-likelihood for logistic regression is given by […] This is also called the cross-entropy error function. — Page 246, Machine Learning: A Probabilistic Perspective, 2012 So I tried that and I found a bit of difference: from…
Easy Points
  • 115
  • 6
0
votes
1 answer

TypeError: Input 'y' of 'Mul' Op has type float32 that does not match type int64 of argument 'x'

after this code i am getting the error in categoricalfocalloss i m not getting whereint64 error is coming def categorical_focal_loss(gamma=2., alpha=.25): def categorical_focal_loss_fixed(y_true, y_pred): y_pred /= K.sum(y_pred,…
AkAnKsHa BaLi
  • 37
  • 1
  • 1
  • 12
0
votes
1 answer

High loss in neural network sequence classification

I am using neural network to classify sequence of length 340 to 8 classes, I am using cross entropy as loss. I am getting very high number for the loss . I am wondering if I did mistake in calculating the loss for each epoch. Or should i use other…
No Na
  • 27
  • 2
  • 7
0
votes
1 answer

RuntimeError: multi-target not supported at /pytorch/aten/src/THCUNN/generic/ClassNLLCriterion.cu:15____

I face that error RuntimeError: multi-target not supported at /pytorch/aten/src/THCUNN/generic/ClassNLLCriterion.cu:15____ My input is binary vector of 340, target is binary vector of 8, For '" loss = criterion(outputs, stat_batch), I got…
No Na
  • 27
  • 2
  • 7
0
votes
2 answers

Why is the class similarity incorrect?

I have created a neural network that classifies data into a large number of classes. I used softmax and categorical cross entropy in Keras. The training was successful and I achieved 95% accuracy. The problem is the following, when I remove the last…
Vladimir
  • 152
  • 1
  • 3
0
votes
0 answers

Training loss is not changing at all while training model

I’m trying to solve a VQA classification problem. my training loss is not changing at all while training the model. I put in comment the CNN model and try to run it with the text only, but still, no change in loss value. I pass through those…
O.H
  • 23
  • 7
0
votes
2 answers

Difference between CrossEntropyLoss and NNLLoss with log_softmax in PyTorch?

When I am building a classifier in PyTorch, I have 2 options to do Using the nn.CrossEntropyLoss without any modification in the model Using the nn.NNLLoss with F.log_softmax added as the last layer in the model So there are two approaches. Now,…
Arkadip Bhattacharya
  • 590
  • 3
  • 13
0
votes
1 answer

Crossentropyloss Pytorch: Targetsize does not match Torchsize

I want to use the Crossentropyloss of pytorch but somehow my code only works with batchsize 2, so i am asuming there is something wrong with the shapes of target and output. I get following error: Value Error: Expected target size (50, 2), got…
Thomas231
  • 45
  • 1
  • 7
0
votes
1 answer

Why don't I get the same result as with tensorflow's method when I write my own expression?

I'm learning logistic regression and I want to calculate what is the value of the cross entropy loss function during minimizing it via gradient descent, but when I use tensorflow's sigmoid_cross_entropy_with_logits function, I get different result…
Ereghard
  • 119
  • 4
1 2 3
…
23 24