In machine learning and information theory, the cross entropy is a measure of distance (inverse similarity) between two probability distributions over the same underlying set of events. Cross entropy is the common choice of the loss function in neural networks for classification tasks.
Questions tagged [cross-entropy]
360 questions
1
vote
1 answer
How to define a “don't care” class in Pytorch?
I have a time series classification task in which I should output a classification of 3 classes for every time stamp t.
All data is labeled per frame.
In the data set are more than 3 classes [which are also imbalanced].
My net should see all samples…
Gulzar
- 23,452
- 27
- 113
- 201
1
vote
2 answers
Why is computing the loss from logits more numerically stable?
In TensorFlow the documentation for SparseCategoricalCrossentropy states that using from_logits=True and therefore excluding the softmax operation in the last model layer is more numerically stable for the loss calculation.
Why is this the case?
marou
- 95
- 1
- 13
1
vote
1 answer
Selecting validation metric for `categorical_crossentropy` in Keras
I am looking at these two questions and documentation:
Whats the output for Keras categorical_accuracy metrics?
Categorical crossentropy need to use categorical_accuracy or accuracy as the metrics in…
Anakin Skywalker
- 2,400
- 5
- 35
- 63
1
vote
1 answer
Intuition behind categorical cross entropy
I'm trying to make categorical cross entropy loss function to better understand intuition behind it.
So far my implementation looks like this:
# Observations
y_true = np.array([[0, 1, 0], [0, 0, 1]])
y_pred = np.array([[0.05, 0.95, 0.05], [0.1, 0.8,…
mikolaj semeniuk
- 2,030
- 15
- 31
1
vote
0 answers
accuracy calculation for a binary classification regression model
can someone tell me why my cross entropy Loss function giving this error:
My accuracy method:
def accuracy(outputs, labels):
_, preds = torch.max(outputs, dim=1)
return torch.tensor(torch.sum(preds == labels).item() / len(preds))
My class…
supremeKAI40
- 11
- 1
1
vote
1 answer
Why am I getting a low error before I did any optimization?
I am using a model training program I have built for a toy example and trying to use it on another example.
The only difference is this model was used for regression, hence I was using MSE as the error criterion, and now it is used for binary…
Guy
- 155
- 11
1
vote
0 answers
How is error calculated in a simple logistic regression neural network?
Below is the following dataframe reporting the results of training a dataset on a binary classification problem:
columns a and b represent x and y respectively and the structure of the neural network is as follows:
the columns h1, h2, h3 and o…
Sunny
- 192
- 9
1
vote
1 answer
How can I calculate cross-entropy on a sigmoid neural network binary outcome?
I'm currently building a NN from scratch where we want to identify based on two input variables (X_1 and X_2) what their output will be (0 or 1). I have 2 hidden layers with sigmoid activation on all neurons however I get stuck when calculating the…
1
vote
1 answer
Pytorch - (Categorical) Cross Entropy Loss using one hot encoding and softmax
I'm looking for a cross entropy loss function in Pytorch that is like the CategoricalCrossEntropyLoss in Tensorflow.
My labels are one hot encoded and the predictions are the outputs of a softmax layer. For example (every sample belongs to one…
joni
- 35
- 2
- 10
1
vote
1 answer
cross entropy loss not equivalent to binary log loss in lgbm
problem trying to solve:
compressing training instances by aggregating label (mean of weighed average) and summing weight based on same feature while keeping binary log loss same as cross entropy loss. Here is an example and test cases of log_loss…
torchflow
- 11
- 2
1
vote
0 answers
Training & validation accuracy increasing & training loss is decreasing - Validation Loss is NaN
I am training a classifier model on cats vs dogs data. The model is a minor variant of ResNet18 & returns a softmax probability for classes. However, I am noticing that the validation loss is majorly NaN whereas training loss is steadily decreasing…
Aditya Mishra
- 1,687
- 2
- 15
- 24
1
vote
1 answer
How is cross entropy loss work in pytorch?
I am experimenting with some of the pytorch codes. With cross entropy loss I found some interesting results and I have used both binary cross entropy loss and cross entropy loss of pytorch.
import torch
import torch.nn as nn
X =…
Eka
- 14,170
- 38
- 128
- 212
1
vote
1 answer
Which loss function should I use if my data is multi-label and my target vectors are not one-hot encoded?
I just read through this link that explains the difference between binary cross-entropy and categorical cross-entropy, and in particular, I had a question about this picture:
The author addressed the multi-label case where your target (or ground…
Malek
- 71
- 3
1
vote
1 answer
Units of the last dense output layer in case of multiple categories
I am currently working on this colab. Task is to classify the sentences into a certain category. So we have a multiple category problem, not binary, like prediction the sentiment of a review (positive / negative) according to certain review…
Stat Tistician
- 813
- 5
- 17
- 45
1
vote
1 answer
PyTorch Multi Class Classification using CrossEntropyLoss - not converging
I am trying to get a simple network to output the probability that a number is in one of three classes. These are, smaller than 1.1, between 1.1 and 1.5 and bigger than 1.5. I am using cross entropy loss with class labels of 0, 1 and 2, but cannot…
Ljackson
- 13
- 4