How can I address individual outputs of a neural network from a custom Tensorflow loss function?

Question

I'd like to implement a custom loss function (specifically for this) for training a NN in Tensorflow, in which NN has two outputs, and the loss function is a formula involving both outputs and the expected output. How can I individually adress those? Examples from tensorflow it looks e.g. like:

def mean_absolute_percentage_error(y_true, y_pred):
  y_pred = ops.convert_to_tensor(y_pred)
  y_true = math_ops.cast(y_true, y_pred.dtype)
  diff = math_ops.abs(
      (y_true - y_pred) / K.clip(math_ops.abs(y_true), K.epsilon(), None))
  return 100. * K.mean(diff, axis=-1)

What exactly are the tensors passed in here as y_true - what shape do they have? I haven't been able to find any documentation about that. All the examples I've seen for custom losses just calculate all outputs at once, whereas I need to pick out individual outputs.

Please note: I cannot define the network into two outputs and define a separate loss for each output, as the Model.compile documentation allows / as it is done here, since the loss function is a function of all three values.

Compute the loss, involving two outputs, as a tensor and then add it to the model using `model.add_loss(...)`. The `loss` argument you specify in `compile` method is applied on each output separately, so it's no use in your case. — today, May 19 '19 at 15:04
@today Thank you! Could you perhaps point me to an example for that? I'm still a newbie to Tensorflow right now. :-) — Dr. Hans-Peter Störr, May 19 '19 at 15:18
This [official Keras example on VAE](https://github.com/keras-team/keras/blob/9d33a024e3893ec2a4a15601261f44725c6715d1/examples/variational_autoencoder.py#L187) might be helpful to get an idea. As you can see there is no `loss` argument specified in `compile` method. Clearly it depends on the architecture of your model, so you may need to adapt it according to your needs. — today, May 19 '19 at 15:21

score 1 · Answer 1 · answered May 20 '19 at 20:00

I didn't find documentation about the shape of the parameters y_pred and y_true, but the following seems to work: to access individual outputs (assuming the output is onedimensional) you can slice on the last axis of the tensor: e.g. y_pred[..., 0:1] is the first output, y_pred[..., 1::2] is every odd numbered output, y_true[..., 0::2] every even numbered label and so forth.

How can I address individual outputs of a neural network from a custom Tensorflow loss function?

1 Answers1