If Y_pred is very far off from Y, the Loss value will be very high. However, if both values are almost similar, the Loss value will be very low. Hence we need to keep a loss function which can penalize a model effectively while it is training on a dataset.
When a neural network is trying to predict a discrete value, we can consider it to be a classification model. This could be a network trying to predict what kind of animal is present in an image, or whether an email is a spam or not.
Questions tagged [loss-function]
1727 questions
7
votes
1 answer
keras combining two losses with adjustable weights
So here is the detail description. I have a keras functional model with two layers with outputs x1 and x2.
x1 = Dense(1,activation='relu')(prev_inp1)
x2 = Dense(2,activation='relu')(prev_inp2)
I need to use these x1 and x2, Merge/add Them and…
madnavs
- 137
- 2
- 8
6
votes
1 answer
Use additional *trainable* variables in Keras/Tensorflow custom loss function
I know how to write a custom loss function in Keras with additional input, not the standard y_true, y_pred pair, see below. My issue is inputting the loss function with a trainable variable (a few of them) which is part of the loss gradient and…
Giora Simchoni
- 3,487
- 3
- 34
- 72
6
votes
1 answer
Keras custom loss function per tensor group
I am writing a custom loss function that requires calculating ratios of predicted values per group. As a simplified example, here is what my Data and model code looks like:
def main():
df = pd.DataFrame(columns=["feature_1", "feature_2",…
DataMan
- 3,115
- 6
- 21
- 36
6
votes
2 answers
How to minimize lasso loss function with scipy.minimize?
Main issue: Why coefficients of Lasso regression are not shrunk to zero with minimization done by scipy.minimize?
I am trying to create Lasso model, using scipy.minimize. However, it is working only when alpha is zero (thus only like basic squared…
Jan Musil
- 508
- 5
- 15
6
votes
0 answers
keras precision recall whith sparse categorical_crossentropy
Is it possible to calculate precision, recall, val_precesion, and val_recall with sparse_categorical_crossentropy loss function? In this post, enter link description here the author suggested to use categorical_crossentropy but in my case, I need to…
midou
- 61
- 3
6
votes
3 answers
keras variational autoencoder loss function
I've read this blog by Keras on VAE implementation, where VAE loss is defined this way:
def vae_loss(x, x_decoded_mean):
xent_loss = objectives.binary_crossentropy(x, x_decoded_mean)
kl_loss = - 0.5 * K.mean(1 + z_log_sigma -…
pnaseri
- 95
- 2
- 9
6
votes
1 answer
Correct Ranking Loss Implementation
I have a multi-label problem and I am trying to implement the Ranking Loss as a custom loss in TensorFlow. (https://arxiv.org/pdf/1312.4894.pdf)
I made a simple CNN with a final Sigmoid layer of activations, to have independent distributions for…
Hichame Yessou
- 2,658
- 2
- 18
- 30
6
votes
3 answers
How to use F-score as error function to train neural networks?
I am pretty new to neural networks. I am training a network in tensorflow, but the number of positive examples is much much less than negative examples in my dataset (it is a medical dataset).
So, I know that F-score calculated from precision and…
Arindam
- 2,116
- 1
- 13
- 10
6
votes
1 answer
Loss function for ordinal multi classification in pytorch
I am a beginner with DNN and pytorch.
I am dealing with a multi-classification problem where my label are encoded into a one-hotted vector, say of dimension D.
To this end, I am using the CrossEntropyLoss. However now I want to modify or change such…
Chutlhu
- 195
- 2
- 10
6
votes
1 answer
How to create a loss function parameter that is dependent on epoch number in Keras?
I have a custom loss function with a hyperparameter alpha that I want to change every 20 epochs over training. The loss function is something like:
def custom_loss(x, x_pred):
loss1 = binary_crossentropy(x, x_pred)
loss2 = (x, x_pred)
…
zucchinifries
- 523
- 6
- 13
6
votes
0 answers
Keras - custom loss function / access 75th percentile element of a tensor
I am trying to implement a slightly modified binary crossentropy loss function for a model in Keras. From Keras, binary_crossentropy is defined as:
def binary_crossentropy(y_true, y_pred):
return K.mean(K.binary_crossentropy(y_true, y_pred),…
atester
- 61
- 3
6
votes
1 answer
Keras training with batches: Is the training loss computed before or after each optimization step?
this is probably a very basic question, however I wasn't able to find an answer to it:
When I train a network with Keras using batches, the console output shows and keeps updating a display of the current loss value of the training set during each…
KiraMichiru
- 958
- 6
- 13
5
votes
1 answer
torch.nn.CrossEntropyLoss over Multiple Batches
I am currently working with torch.nn.CrossEntropyLoss. As far as I know, it is common to compute the loss batch-wise. However, is there a possibility to compute the loss over multiple batches?
More concretely, assume we are given the data
import…
Vivian
- 335
- 2
- 10
5
votes
2 answers
Can we use multiple loss functions in same layer?
Can we use mulitple loss function in this architecture:
I have two different type of loss functions and want to use it on last layer [Output]
loss functions :
binary_crossentropy
custom loss function
Can we do that?
Pawan bisht
- 183
- 1
- 9
5
votes
2 answers
What should I use as target vector when I use BinaryCrossentropy(from_logits=True) in tensorflow.keras
I have a multi-label classification in which each target is a vector of ones and zeros not mutually exclusive (for the sake of clarity, my target is something like [0, 1, 0, 0, 1, 1, ... ]).
My understanding so far is:
I should use a binary…
Luca
- 1,610
- 1
- 19
- 30