In machine learning and information theory, the cross entropy is a measure of distance (inverse similarity) between two probability distributions over the same underlying set of events. Cross entropy is the common choice of the loss function in neural networks for classification tasks.
Questions tagged [cross-entropy]
360 questions
0
votes
0 answers
Training accuracy not improve in CNN
My model is a SegNet with cross-entrophy loss. It is for liver and liver vessel medical segmentation.
And my problem is, when I use 725 cases for input, the accuracy increase normally.
The result of liver is good, but the result of vessel still need…
chou
- 1
- 1
0
votes
0 answers
Computing loss in Language model
def compute_loss(ouputs, y, criterion):
"""Compute the loss given the ouput, ground truth y, and the criterion function.
Hint: criterion should be cross entropy.
Hint: the input is a list of tensors, each has shape as (batch_size, vocab_size)
…
Ethereal soul
- 709
- 1
- 7
- 20
0
votes
0 answers
Bert only trains for positive labels and not for negative labels
I am fine-tuning BertModel for Sentiment Analysis with Crossentropy loss.
But it only trains if labels are positive in nature but when training for negative labels it throws an error.
For ex- it labels are 0,1,2 it trains perfectly.
But if labels…
anuja
- 170
- 1
- 2
- 11
0
votes
1 answer
Cross entopy question for semantic segmentation procedure
How to use cross entropy for float images and labels?
I'm studying Semantic segmentation with Cityscapes data.
When I use
torchivision.transfrom.ToTensor()
My Images and labels data are transform the range from [0~255] to [0~1]. Because I used…
0
votes
1 answer
How to find Entropy or KL divergence between two data?
I have two datasets which are same shape: (576, 450, 5) where 576 is the number of examples, 450 is the time points and 5 is the number of channels.
I want to calculate entropy and KL-divergence between these two datas. But I know that the entropy…
ali
- 119
- 1
- 3
- 15
0
votes
0 answers
Keras Custom Loss function Categorical Cross Entropy Loss
I have been trying to write a custom loss function in Keras to include reward in Reinforcement Learning.
The model takes current state image and previous action as input. The previous action is concatenated at a later stage in the model.…
0
votes
1 answer
BERT Transformer model gives an error for multiclass classification
I am trying to train a sentiment analysis model with 5 classes (1-Very Negative, 2-Negative, 3-Neutral, 4-Positive, 5-Very Positive) with the BERT model.
from transformers import BertTokenizer, TFBertForSequenceClassification
from transformers…
Kay
- 567
- 3
- 6
- 15
0
votes
1 answer
Transformer Model only predict Start or End Tokens
So I've been trying to build and train a Transformer Model from scratch for empathetic dialogue generation tasks and currently I'm struggling with the training process since the model only seems to predict START and END tokens in the final output…
Tarush
- 1
- 1
0
votes
1 answer
Custom loss for single-label, multi-class problem
I have a single-label, multi-class classification problem, i.e., a given sample is in exactly one class (say, class 3), but for training purposes, predicting class 2 or 5 is still okay to not penalise the model that heavily.
For example, the ground…
helloworld
- 150
- 1
- 7
0
votes
1 answer
Average cross entropy for 3D tensors
I have an output tensor (both target and predicted) of dimension (32 x 8 x 5000). Here, the batch size is 32, the number of classes is 5000 and the number of points per batch is 8. I want to calculate CELoss on this in such a way that, the loss is…
helloworld
- 150
- 1
- 7
0
votes
2 answers
Multi-target loss recommendations
I'm working on a classification problem. The number of classes is 5. I have a ground truth vector that has the shape (3) instead of 1. The values in this target vector are the possible classes and the predicted vector is of the shape (1x5) which…
helloworld
- 150
- 1
- 7
0
votes
0 answers
Error when using binary_cross_entropy_with_logits: ValueError: Target size must be the same as input size
I have a very simple GNN class (stripped to the bone to create a MRE):
import torch
from torch.nn import Linear
import torch.nn.functional as F
from torch_geometric.nn.conv import GraphConv
from torch_geometric.nn.pool import global_add_pool
class…
DeltaIV
- 4,773
- 12
- 39
- 86
0
votes
0 answers
"warning: basinhopping: local minimization failure" when doing maximum entropy optimization
I'm minimizing a multiple-variable function using scipy.optimize.basinhopping as part of maximum entropy optimization. It works but occasionally I get local minimization failure warning messages. I'm trying to figure out whether they can be…
Neil Fazel
- 1
- 2
0
votes
0 answers
ValueError: classes should include all valid labels that can be in y using compute_class_Weight
Yes there was a similar thread on the question and those solutions are not working for me.
The error is as follow:
NMK
- 11
- 1
0
votes
0 answers
RuntimeError: only batches of spatial targets supported (3D tensors) but got targets of size: : [32]
I'm trying to break down the error I'm getting from the loss function. My train function is below:
model = model.to(device)
optimizer = torch.optim.SGD(model.parameters(), lr=LR)
criterion = nn.CrossEntropyLoss()
for epoch in range(2):
for idx,…
zampoan
- 57
- 7