6

I would like to create a custom loss function that has a weight term that's updated based on what epoch I'm in.

For example: Let's say I have a loss function which has a beta weight, where beta increases over the first 20 epochs...

def custom_loss(x, x_pred): 
    loss1 = objectives.binary_crossentropy(x, x_pred)
    loss2 = objectives.mse(x, x_pred)
    return (beta*current_epoch/20) * loss1 + loss2

How could I implement something like this into a keras loss function?

foboi1122
  • 1,727
  • 4
  • 19
  • 36

1 Answers1

0

Looking at their documentation they mention that you can use theano/Tf symbolic functions that return a scalar for each data point. So you could do something like this 

loss = tf.contrib.losses.softmax_cross_entropy(x, x_pred) * 
       (beta * current_epoch / 20 ) +  
       tf.contrib.losses.mean_squared_error

You would have to pass x and x_pred as x and x_pred as tf.placeholders I think for model creation you could use keras but then again you would have to run the computational graph with sess.run()

References: https://blog.keras.io/keras-as-a-simplified-interface-to-tensorflow-tutorial.html#using-keras-models-with-tensorflow

Kris
  • 518
  • 3
  • 13