4

I'm starting out with GANs and I am training a DC-GAN on MNIST dataset. I want to evaluate my model using Frechet Inception Distance (FID).

  1. Since Inception network is not trained to classify MNIST digits, can I use any simple MNIST classifier or are there any conditions on what kind of classifier I need to use? Or should I use Inception net only? I have few other questions
  2. Does it make sense to compute FID for MNIST GAN?
  3. How many images from real dataset should be used while computing FID
  4. For a classifier I'm using, I'm getting FID in the order of 10^6. Is the value okay or is something horribly wrong?

If you can answer any of these questions, even partially, that would be of immense help to me. Thanks!

Nagabhushan S N
  • 6,407
  • 8
  • 44
  • 87

2 Answers2

2

Model trained on Mnist dont do well on FID computation. As far as I can tell, major reasons are data distribution is too narrow(Gan images are too far from distribution model is trained on) and model is not deep enough to learn a lot of feature variation.

Training a few-convolutional layers model gives 10^6 values on FID. To test the above hypothesis, just adding L2 regularization, the FID values dropped to around 3k(confirming to data distribution being narrow), however the FID value dont improve as GAN training goes on. :(.

Finally, directly computing FID values from Inception network gives a nice plot as images becomes better. [Note:- You need to rescale mnist image and convert to RGB by repeating one channel 3 times. Make sure real image and generated image have same intensity scales.]

FID and inception score

saurabheights
  • 3,967
  • 2
  • 31
  • 50
1

you can refer this. Use a auto-encoder trained on MNIST and the bottleneck activations as the features as explained here

manoj surya k
  • 94
  • 3
  • 4
    Welcome to stackoverflow. Best practice is to include key details in your answer, so that it is helpful even if the content you have linked to gets changed. – Simon.S.A. Sep 06 '19 at 06:03
  • Did u try autoencoder approach? [AFAIK, this approach might not have worked] Feel free to disagree :), interested myself on this – saurabheights Jan 06 '20 at 18:00