In machine learning and information theory, the cross entropy is a measure of distance (inverse similarity) between two probability distributions over the same underlying set of events. Cross entropy is the common choice of the loss function in neural networks for classification tasks.
Questions tagged [cross-entropy]
360 questions
1
vote
1 answer
Modified PyTorch loss function BCEWithLogitsLoss returns NaNs
I'm trying to solve a binary classification problem (target=0 and target=1) with an exception:
Some of my labels are classified as target=0.5 on purpose, and I wish to have zero loss for either classifying it as 0 or 1 (i.e both classes are…
Shlomi Shmuel
- 21
- 1
- 7
1
vote
1 answer
How to label the loss values in Keras binary-crossentropy model
I have a keras model for my data X. The code used is:
X=np.array(data[['tags1','prx1','prxcol1','p1','p2','p3']].values)
t=np.array(data.read.values)
n=np.array(data.read.values)
import keras
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'…
pablo
- 385
- 2
- 14
1
vote
0 answers
Target 40 is out of bounds for nn.CrossEntropyLoss()
I create a custom image data set like:
from torch.utils.data.dataset import Dataset
from PIL import Image
import torchvision
from torchvision import datasets, models, transforms
import numpy as np
class MyCustomDataset(Dataset):
def…
wangmyde
- 77
- 8
1
vote
0 answers
Softmax cross entropy loss for single batch in python
i want to ask how to implement cross entropy loss for single batch in neural network, where the equation is:
this is my code for cross entropy only for single example:
def softmax_cross_entropy(y_true, y_pred):
softmax_cross_entropy_loss_single…
thenoirlatte
- 341
- 3
- 19
1
vote
2 answers
Pytorch: multi-target error with CrossEntropyLoss
So I was training a Conv. Neural Network. Following are the essential details:
original label dim = torch.Size([64, 1])
output from the net dim = torch.Size([64, 2])
loss type = nn.CrossEntropyLoss()
error = RuntimeError: multi-target not…
Parthik B.
- 472
- 1
- 5
- 15
1
vote
1 answer
Multi class classifcation with Pytorch
I'm new with Pytorch and I need a clarification on multiclass classification.
I'm fine-tuning the DenseNet neural network, so it can recognize 3 different classes.
Because it's a multiclass problem, I have to replace the classification layer in this…
maccN
- 189
- 1
- 4
- 13
1
vote
0 answers
validation AUC goes up but crossentropy rises aswell
So I have a neural network using cross-entropy (CE) as loss-function, its a binary classification.
I am using the AUC as a validation metric, and when I plot training and validation error (i.e the CE), the is the training falling, the validation is…
CutePoison
- 4,679
- 5
- 28
- 63
1
vote
0 answers
Cross-entropy rises while AUC rises (pytorch)
I have a feed-forward neural network and a binary classification problem.
Defining the loss function as
def cross_entropy(ys,ts):
cross_entropy = -torch.sum(ts * torch.log(ys+0.00001) + (1-ts)*torch.log(1-ys+0.00001))
return…
CutePoison
- 4,679
- 5
- 28
- 63
1
vote
2 answers
How to write custom CrossEntropyLoss
I am learning Logistic Regression within Pytorch and to better understand I am defining a custom CrossEntropyLoss as below:
def softmax(x):
exp_x = torch.exp(x)
sum_x = torch.sum(exp_x, dim=1, keepdim=True)
return exp_x/sum_x
def…
A.E
- 997
- 1
- 16
- 33
1
vote
1 answer
Binary cross entropy Vs categorical cross entropy with 2 classes
When considering the problem of classifying an input to one of 2 classes, 99% of the examples I saw used a NN with a single output and sigmoid as their activation followed by a binary cross-entropy loss. Another option that I thought of is having…
MRm
- 517
- 2
- 14
1
vote
1 answer
cross entropy loss rises and falls periodically on image-net
I am training my mobilenet v3 with tfrecords produced by tensorflow model. The training loss w.r.t steps is plotted below. Unit length in x axis is 20k steps (2 epochs approximately due to batch size=128 and 1281167 samples totally).
I exponential…
Euphoria Yang
- 81
- 7
1
vote
0 answers
Binary classification predicting the same class
Good morning, I'm implementing a binary classification model of two exclusive one-hot encoded labels. I finally got to the stage that the program outputs something, but sadly enough it is predicting the same class the whole time. At one time it did…
MarcMiranda
- 23
- 5
1
vote
2 answers
shape of output tensor by keras.losses.binary_crossentropy
I want to implement a custom loss function in keras based on binary_crossEntropy. I have question about shape of output tnesor of Keras.losses.binary_crossentropy. I expect that it should be a 1D tensor with length of batch_size. but it returns a…
Jafar Gh
- 137
- 2
- 12
1
vote
0 answers
binary classification, xentropy mismatch , invalid argument ( Received a label value of 1 which is outside the valid range of [0, 1) )
I'm working on a Deep neural Network for text-classification but I got a problem with my xentropy.
I'm following a course with multiclass classification and I try to adapt it to my binary classification problem.
The course used softmax for…
Benech17
- 89
- 11
1
vote
1 answer
Cross Entropy Error remains unchanged for various values
I am using Cross Entropy with Softmax as loss function for my neural network.
The cross entropy function I have written is as follows:
def CrossEntropy(calculated,desired):
sum=0
n=len(calculated)
for i in range(0,n):
…
pr22
- 179
- 1
- 2
- 9