Stack Overflow
Stack Overflow

Questions tagged [cross-entropy]

In machine learning and information theory, the cross entropy is a measure of distance (inverse similarity) between two probability distributions over the same underlying set of events. Cross entropy is the common choice of the loss function in neural networks for classification tasks.

360 questions
1
vote
0 answers

Ignore/Mask certain labels Softmax Cross-entropy

In trying implement the cross entropy loss with l2 regularization described here A Fast and Accurate Dependency Parser using Neural Networks, I get the error: ValueError: Cannot feed value of shape (48,) for Tensor u'Placeholder_2:0', which has…
dreamin
  • 187
  • 1
  • 3
  • 12
1
vote
2 answers

Python - Low accuracy with low loss with tensorflow

I'm building a simple neural network that takes 3 values and gives 2 outputs. I'm getting an accuracy of 67.5% and an average cost of 0.05 I have a training dataset of 1000 examples and 500 testing examples. I plan on making a larger dataset in the…
Blair Burns
  • 41
  • 1
  • 7
1
vote
1 answer

Is it possible to implement a loss function that prioritizes the correct answer being in the top k probabilities?

I am working on an multi-class image recognition problem. The task is to have the correct answer being in the top 3 output probabilities. So I was thinking that maybe there exists a clever cost function that prioritizes the correct answer being in…
MLearner
  • 164
  • 1
  • 1
  • 9
1
vote
1 answer

scaling the sparse_softmax_cross_entropy_with_logits

How could I scale gradients where the loss comes from sparse_softmax_cross_entropy_with_logits. For example, I was trying to divide by 128 as below, but I found error: new_gradients = [(grad/128, var) for (grad, var) in gradients] TypeError:…
Mostofa Patwary
  • 11
  • 2
1
vote
1 answer

Tensorflow ValueError: Only call `sparse_softmax_cross_entropy_with_logits` with named arguments

When calling the following method: losses = [tf.nn.sparse_softmax_cross_entropy_with_logits(logits, labels) for logits, labels in zip(logits_series,labels_series)] I receive the following ValueError: ValueError: Only call…
Glennismade
  • 117
  • 2
  • 14
1
vote
0 answers

Individual weights per class for each sample in Keras

In Keras (tensorflow backend) I can weight the classes via a parameter, and there is another parameter that allows the weighting of the samples (as it was referenced in this question.) In my case, I need to do both simultaneously: each sample should…
Nickpick
  • 6,163
  • 16
  • 65
  • 116
1
vote
1 answer

Tensorflow - Using tf.losses.hinge_loss causing Shapes Incompatible error

My current code using sparse_softmax_cross_entropy works fine. loss_normal = ( tf.reduce_mean(tf.losses .sparse_softmax_cross_entropy(labels=labels, logits=logits, …
Allen Qin
  • 19,507
  • 8
  • 51
  • 67
1
vote
1 answer

tensorflow softmax_cross_entropy code

Since the source code of tf.nn.softmax_cross_entropy_with_logits in gen_nn_ops is hidden, could anyone perhaps explain me how tensorflow compute the cross entropy after Softmax. I mean, after softmax it might output 0 because of precision which will…
Nan
  • 299
  • 2
  • 3
  • 9
1
vote
1 answer

Is it possible to convert tensorflow code to theano code?

I have a function that uses some tensorflow functions. I need this function in Theano because on the Platform i want to use this code there is only Theano installed and not tensorflow. I am working mainly with Keras so tensorflow is quite cryptic…
Mimi Müller
  • 416
  • 8
  • 25
1
vote
1 answer

Neural Network loss starts increasing while acc is increasing on both train/val data sets

For past few days I have been debugging my NN but I can't find an issue. I've created total raw implementation of multi-layer perceptron for identifying MNIST dataset images. Network seems to learn because after train cycle test data accuracy is…
F1sher
  • 7,140
  • 11
  • 48
  • 85
1
vote
1 answer

Cross-entropy implementation in Caffe

Looking at the source code in sigmoid_cross_entropy_loss_layer.cpp, which is the source code for Cross-Entropy loss function in caffe, I noticed that the code for the actual error value is for (int i = 0; i < count; ++i) { loss -=…
Alex
  • 944
  • 4
  • 15
  • 28
1
vote
0 answers

Importance of a Random Variable using Entropy or other method

I have a two-dimensional random vector x = [x1, x2]T with a known joint probability density function (PDF). The PDF is non-Gaussian and the two entries of the random vector are statistically dependent. I need to show that for example x1 is more…
user3016420
  • 11
  • 1
1
vote
0 answers

What is the Optimum Cross Entropy Error for Binary Classification

Cross Entropy Error is the standard loss function for a number of classification algorithms. I would like to know what is the optimum cross entropy error I should expect for binary classification?
rocklegend
  • 81
  • 11
1
vote
0 answers

How can I normalize my loss function in relation to the number of elements in a class in Tensorflow?

In an image segmentation problem, we will usually get a background label that dominates the mask and the object(s) occupying only a small area in the mask. As such, during training, I have observed that my neural network does very well in…
Ray.R.Chua
  • 777
  • 3
  • 8
  • 27
1
vote
1 answer

Need help understanding the Caffe code for SigmoidCrossEntropyLossLayer for multi-label loss

I need help in understanding the Caffe function, SigmoidCrossEntropyLossLayer, which is the cross-entropy error with logistic activation. Basically, the cross-entropy error for a single example with N independent targets is denoted as: -…
auro
  • 1,079
  • 1
  • 10
  • 22
1 2 3
…
23 24