If Y_pred is very far off from Y, the Loss value will be very high. However, if both values are almost similar, the Loss value will be very low. Hence we need to keep a loss function which can penalize a model effectively while it is training on a dataset.
When a neural network is trying to predict a discrete value, we can consider it to be a classification model. This could be a network trying to predict what kind of animal is present in an image, or whether an email is a spam or not.
Questions tagged [loss-function]
1727 questions
10
votes
1 answer
How can I use TensorFlow's sampled softmax loss function in a Keras model?
I'm training a language model in Keras and would like to speed up training by using sampled softmax as the final activation function in my network. From the TF docs, it looks like I need to supply arguments for weights and biases, but I'm unsure of…
kylerthecreator
- 1,509
- 3
- 15
- 32
10
votes
4 answers
How to do point-wise categorical crossentropy loss in Keras?
I have a network that produces a 4D output tensor where the value at each position in spatial dimensions (~pixel) is to be interpreted as the class probabilities for that position. In other words, the output is (num_batches, height, width,…
Alex I
- 19,689
- 9
- 86
- 158
9
votes
1 answer
Understanding of Pytorch NLLLOSS
PyTorch's negative log-likelihood loss, nn.NLLLoss is defined as:
So, if the loss is calculated with the standard weight of one in a single batch the formula for the loss is always:
-1 * (prediction of model for correct class)
Example:
Correct…
Friseur
- 103
- 1
- 7
9
votes
2 answers
Resume training with different loss function
I want to implement a two-step learning process where:
pre-train a model for a few epochs using the loss function loss_1
change the loss function to loss_2 and continue the training for fine-tuning
Currently, my approach…
dave
- 113
- 8
9
votes
1 answer
Keras: what does class_weight actually try to balance?
My data has extreme class imbalance. About 99.99% of samples are negatives; the positives are (roughly) equally divided among three other classes. I think the models I'm training are just predicting the majority class basically all the time. For…
Randoms
- 2,110
- 2
- 20
- 31
8
votes
1 answer
Can SigmoidFocalCrossEntropy in Tensorflow (tf-addons) be used in Multiclass Classification? ( What is the right way)?
Focal Loss given in Tensorflow is used for class imbalance. For Binary class classification, there are a lots of codes available but for Multiclass classification, a very little help is there. I ran the code with One Hot Encoded target variables of…
Deshwal
- 3,436
- 4
- 35
- 94
8
votes
2 answers
Implementing Binary Cross Entropy loss gives different answer than Tensorflow's
I am implementing the Binary Cross-Entropy loss function with Raw python but it gives me a very different answer than Tensorflow.
This is the answer I got from Tensorflow:-
import numpy as np
from tensorflow.keras.losses import…
user12188405
8
votes
3 answers
Custom loss function with weights in Keras
I'm new with neural networks. I wanted to make a custom loss function in TensorFlow, but I need to get a vector of weights, so I did it in this way:
def my_loss(weights):
def custom_loss(y, y_pred):
return weights*(y - y_pred)
return…
Michael Moretti
- 247
- 4
- 15
8
votes
2 answers
How to replace loss function during training tensorflow.keras
I want to replace the loss function related to my neural network during training, this is the network:
model = tensorflow.keras.models.Sequential()
model.add(tensorflow.keras.layers.Conv2D(32, kernel_size=(3, 3), activation="relu",…
Francesco Scala
- 201
- 3
- 15
8
votes
1 answer
Why am I getting different values between loss functions and metrics in TensorFlow Keras?
In my CNN training using TensorFlow, I am using Keras.losses.poisson as a loss function. Now, I like to calculate many metrics alongside that loss function, and I am observing that Keras.metrics.poisson gives different results - although the two are…
bers
- 4,817
- 2
- 40
- 59
8
votes
2 answers
Custom Loss Function in R Keras
I want to calculate weighted mean squared error, where weights is one vector in the data. I wrote a custom code based on the suggestions available on stack overflow.
The function is provided below:
weighted_mse <- function(y_true,…
Sumit
- 2,242
- 4
- 25
- 43
8
votes
2 answers
Implementing custom loss function in keras with condition
I need some help with keras loss function. I have been implementing custom loss function on keras with Tensorflow backend.
I have implemented the custom loss function in numpy but it would be great if it could be translated into keras loss…
Black Mask
- 387
- 1
- 3
- 16
8
votes
1 answer
Keras: Weighted Binary Crossentropy Implementation
I'm new to Keras (and ML in general) and I'm trying to train a binary classifier. I'm using weighted binary cross entropy as a loss function but I am unsure how I can test if my implementation is correct.
Is this an accurate implementation of…
Stefan Iarca
- 152
- 1
- 7
7
votes
2 answers
Custom loss in XGBoost is not updating
Context
I am trying to use a custom loss function for an XGBoost binary classifier.
The idea was to implement in XGBoost the soft-Fbeta loss, which I read about here. Simply put: instead of using the standard logloss, use a loss function that…
GiacomoP
- 83
- 3
7
votes
2 answers
Interpreting the effect of LK Norm with different orders on training machine learning model with the presence of outliers
( Both the RMSE and the MAE are ways to measure the distance between two vectors: the vector of predictions and the vector of target values. Various distance measures, or norms, are possible. Generally speaking, calculating the size or length of a…
I. A
- 2,252
- 26
- 65