2

I am training a VAE using the Keras tutorial: https://keras.io/examples/generative/vae/. This involves creating a VAE class and specifying a custom training procedure, which is described more here: https://keras.io/guides/customizing_what_happens_in_fit/. After creating an encoder and decoder as specified in the tutorial and training a model, I create the VAE model through the following:

vae = VAE(encoder, decoder) 
vae.compile(optimizer=keras.optimizers.Adam())

vae.fit(x=x_train, y=None, 
        epochs=epochs, 
        batch_size=batch_size, 
        verbose=False,
        validation_data=(x_test, None))

I want to evaluate the model on separate datasets after training (since I have multiple evaluation datasets, I am not using it as validation_data). However, when I try to run vae.evaluate(data), it returns an empty list [].

Note: I am able to obtain training and validation metrics easily through vae.history.history, but the issue is when I try to evaluate after training. But when I try to return the metrics vae.metrics it also returns an empty list. How do I get model.evaluate to work with a custom training procedure that returns a dict of loss metrics? Do I need to define something custom for evaluate?

Here is how the VAE class is defined. More details can be found in the tutorial above.

class VAE(keras.Model):
    def __init__(self, encoder, decoder, **kwargs):
        super(VAE, self).__init__(**kwargs)
        self.encoder = encoder
        self.decoder = decoder

    def train_step(self, data):
        if isinstance(data, tuple):
            data = data[0]
        with tf.GradientTape() as tape:
            z_mean, z_log_var, z = encoder(data)
            reconstruction = decoder(z)
            reconstruction_loss = tf.reduce_mean(
                keras.losses.binary_crossentropy(data, reconstruction)
            )
            reconstruction_loss *= 28 * 28
            kl_loss = 1 + z_log_var - tf.square(z_mean) - tf.exp(z_log_var)
            kl_loss = tf.reduce_mean(kl_loss)
            kl_loss *= -0.5
            total_loss = reconstruction_loss + kl_loss
        grads = tape.gradient(total_loss, self.trainable_weights)
        self.optimizer.apply_gradients(zip(grads, self.trainable_weights))
        return {
            "loss": total_loss,
            "reconstruction_loss": reconstruction_loss,
            "kl_loss": kl_loss,
        }
Jane Sully
  • 3,137
  • 10
  • 48
  • 87
  • per the article by fchollet which you provided, you'll need to implement/override method `test_step`. I tried that but am still getting an empty list! – pangyuteng Nov 15 '20 at 14:52
  • evaluate returns numbers now, once loss trackers are implemented and `metrics` method is overriden. – pangyuteng Nov 15 '20 at 15:35

0 Answers0