0

I want to calculate the gradients, my model is a little bit difficult, it can be represented as the following loss function:

loss=label1*f1(x)+label2*f2(x)+label3*f3(x)

x is the feature of model, f1 is one model which is predicting one logit, you can see that there are three models and I want to sum them together. In order for easy understanding, loss function can be written as 

loss=-tf.reduce_sum(label * tf.log(x), axis=1)

but I have three different lables and different models and same feature. I find tf.GradientTape is a easy way to calculate the gradients(https://www.tensorflow.org/tutorials/eager/custom_training_walkthrough), but they use 

loss_value, grads = grad(model, features, labels)

there is only one model ,one feature list and one labels list.but how to solve the above multiclass problem?

andy
  • 1,951
  • 5
  • 16
  • 30
  • Why you don't simply concatenate f1(x), f2(x) , f3(x) to create a single tensor with your logits and then use tf.nn.sigmoid_cross_entropy_with_logits to compute the cross_entropy , and then you can use tf.reduce_mean(cross_entropy) as your loss. Concerning the model, you can arrange yourself to have "three models" but as one model. – abcdaire Jan 18 '19 at 01:49
  • can I use t1=tf.nn.sigmoid_cross_entropy_with_logits(labels=label1, logits=x) first, then concatenate with t= tf.concat([t1, t2,t3], 1),then tf.reduce_mean(t)? Because my three different models in different loops – andy Jan 18 '19 at 07:20
  • I think it's equivalent – abcdaire Jan 18 '19 at 09:52

0 Answers0