I've been attempting to implement my own GAN with some limited successes. I tweaked a bit of how they trained it in the tutorial and was wondering if this more drastic change is viable. The first change I made was to make the discriminator classify n+1 classes where n could be, for example 10 in MNIST, and the n+1th class is the fake class. The discriminator was an imported architecture I made from scratch for a really good classifier. Then my GAN will have the "opposite" of the traditional NLLLoss.
This is the tricky non-traditional calculation part. So because I have a softmax in the last layer of my discriminator, my output from the discriminator will always be from 0 to 1. So I can create a custom loss function for the generator to be a horizontal flip of the NLLLoss which tries to make sure the discriminator does not classify the fakes as the n+1th class. The idea is that I don't care what class the fakes are classified as long as they're not the n+1th class. This behavior of misclassification is what I want to maximize for the generator. Here is the function I plotted on desmos to give some visualization:
https://www.desmos.com/calculator/6gdqs28ihk
my actual code for the generator loss function is the code below while my discriminator loss function is the traditional NLLLoss
loss_G = torch.mean(-torch.log(1 - outputG.float()[:,classes]))
Please let me know if this is completely wrong or there is an easier way.
