In machine learning and information theory, the cross entropy is a measure of distance (inverse similarity) between two probability distributions over the same underlying set of events. Cross entropy is the common choice of the loss function in neural networks for classification tasks.
Questions tagged [cross-entropy]
360 questions
0
votes
1 answer
How can I choose the wights of CrossEntropy loss?
I am dealing with a semantic segmentation model which classifies each pixel in 3 classes: background, liver and tumor. I am using the classic UNET architecture with CrossEntropyLoss from the PyTorch implementation.
Since the dataset is imbalance…
Fredi
- 9
- 2
0
votes
1 answer
Why does my MLP model's loss explode when using softmax and cross entropy in Python?
I am writing an NLP model from scratch in Python, using only NumPy for most of the functions.
import numpy as np
# my loss and activation functions
def relu(x):
return np.maximum(0, x)
def relu_prime(x):
return np.where(x > 0, 1, 0)
def…
Capta1n_n9m0
- 18
- 4
0
votes
0 answers
Cross entropy value higher than entropy
I'm getting a result for entropy that is higher than for cross-entropy, which shouldn't be possible:
import numpy as np
# Define some sample data
y_true = np.array([0.10, 0.40, 0.50])
y_pred = np.array([0.20, 0.40, 0.45])
# cross entropy…
tensai
- 51
- 7
0
votes
1 answer
PyTorch CrossEntropyLoss documentation example crashes
To make sure I'm using PyTorch CrossEntropyLoss correctly, I'm trying the examples from the documentation: https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html
However, the first example (target with class indices) doesn't seem…
rwallace
- 31,405
- 40
- 123
- 242
0
votes
0 answers
Shape mismatch between logits and labels for computing loss on 3D data in GCN
RuntimeError: Expected target size [25, 10], got [25].
I have a sensors data and I want to perform node level classification using GCN. I have k nodes (sensors). Each node has m observations (node features) and each observation is a n dimensional…
yari
- 1
- 1
0
votes
0 answers
Getting Error: TypeError: cross_entropy_loss(): argument 'target' (position 2) must be Tensor, not tuple
I am working on a CNN multi-class classification of different concentrations (10uM, 30uM, etc.) I create my dataset to include the images as the features and the concentrations as labels. Note that the concentrations are left as a string. When…
Zelreedy
- 15
- 4
0
votes
0 answers
Cross Entropy Loss Gets Negative Values In Training Transformer Model
I built my Transformer model for recovering text. In detail, the source text may contain some redundant, missing or wrong words, my model have to correct as many as possible these words. Moreover, I just want my model learn embedding of the correct…
0
votes
0 answers
Role that learning_rate plays in the reproducibility of the model in PyTorch models
I have a Bayesian neural netowrk which is implemented in PyTorch and is trained via a ELBO loss. I have faced some reproducibility issues even when I have the same seed and I set the following code:
# python
seed =…
samsambakster
- 43
- 6
0
votes
1 answer
Getting wrong output while calculating Cross entropy loss using pytorch
Getting wrong output while calculating Cross entropy loss using pytorch
Hi guys ,
I calculated the cross entropy loss using pytorch , Input = torch.tensor([[1.,0.0,0.0],[1.,0.0,0.0]]) ,label = torch.tensor([0, 0]) . The output must be 0 but I got (…
Sukesh Ram
- 9
- 1
0
votes
0 answers
How to solve "TypeError: cross_entropy_loss(): argument 'input' (position 1) must be Tensor, not tuple"?
I'm trying to run the DeepCrack model, a CNN to find a pavement crack detection model. But I'm getting this error. I found the following error. I understood that the problem is in Outputs = model(image) here because my model is returning a tuple…
NOWORRIES
- 1
- 2
0
votes
1 answer
Does LightGBM binary classifier with (AUC-based) early-stopping rounds take log loss as objective function, or use AUC in optimization algorithm?
I have a LightGBM gradient boosting model for a binary classification task. The LightGBM parameters specification state that the objective function of binary classification is the log loss (cross entropy). Hence, I understand that is the objective…
Jim R.
- 3
- 1
0
votes
0 answers
keras binary_crossentropy loss works well even with a shape mismatch in y_true and y_pred, while it throws error if from_logits = True?
import numpy as np
fake_preds = np.random.rand(10,768) # assuming predictions of some model (10 examples)
fake_labels = np.random.rand(10,1) # fake labels for 10 examples
loss1 =…
Rahul Ohlan
- 1
- 2
0
votes
0 answers
Should I use from_logits on AUC keras metric with sigmoid activation function?
I'm currently implementing a convolutional neural network model that outputs binary classification (true or false), and the labels are all either 0 or 1. When using the "sigmoid" activation function for the final dense layer, I was wondering whether…
0
votes
1 answer
What is the correct way to penalize one prediction more over another?
I have a BERT-based sequence classification model that takes as an input 4 strings and out 2 labels for each one:
my_input = [string_1, string_2, string_3, string_4]
out_logits = model(my_input).logits
out_softmax =…
Penguin
- 1,923
- 3
- 21
- 51
0
votes
0 answers
User defined cross entropy and built-in cross entropy of KERAS gives same loss but different gradients
When i train a simple NN architecture with cross entropy i get same loss when using the built-in keras crossentropy loss and user defined crossentropy while the gradients are different for the different implementation ...
Important to mention that…
razisamuely
- 1
- 1