Defining loss function for autoencoder in Tensorflow

Question

I am trying to create autoencoder (CVAE) on similar lines like one given here: Use Conditional Variational Autoencoder for Regression (CVAE). However, in vae_loss() and in KL_loss(), different variables (l_sigma, mu) are used than what these functions take in. Running the code like this gives error TypeError missing 2 required positional arguments My question is what is correct way to pass the required variables, in this case 4 - l_sigma, mu, y_true, y_pred, to the loss functions through cvae.compile() and cvae.fit() ? There seems another way https://github.com/PacktPublishing/Advanced-Deep-Learning-with-Keras/blob/master/chapter8-vae/cvae-cnn-mnist-8.2.1.py#L267 to define the loss function giving the required 4 variables. Any idea?

Answer 1

To add hyperparameters to a custom loss function using Tensorflow you have to create a wrapper function that takes the hyperparameters, so you can try define your custom loss function as follow:

def vae_loss_with_hyperparameters(l_sigma, mu):
    def vae_loss(y_true, y_pred):
        recon = K.sum(K.binary_crossentropy(y_true, y_pred), axis=-1)
        kl = 0.5 * K.sum(K.exp(l_sigma) + K.square(mu) - 1. - l_sigma, axis=-1)
        return recon + kl
    return vae_loss

After, you can call the compile() method like that: cvae.compile(loss=vae_loss_with_hyperparameters(l_sigma=..., mu=...))

Let me know if it works.