Thank you so much explaining GAN in very simple way. It will help me a lot on my thesis!!

@SebastianRaschka

2 жыл бұрын

Thanks for the kind words, and best of luck with your thesis!

why discriminator loss is taken as the average of real_loss and fake_loss and not just their sum?

When you are training the generator where do you freeze the discriminator in the code for MNIST dataset ?

@SebastianRaschka

2 жыл бұрын

Good question! It's actually "implicit" so it's not easy to see. The relevant lines are: # get discriminator loss on fake images with flipped labels discr_pred_fake = model.discriminator_forward(fake_images).view(-1) gener_loss = loss_fn(discr_pred_fake, flipped_fake_labels) gener_loss.backward() optimizer_gen.step() So, here the "freezing" is not something we have to do explicitly, but it's automatically taken care of since we have two separate optimizers. I.e., we defined the generator loss as follows: optim_gen = torch.optim.Adam(model.generator.parameters(), betas=(0.5, 0.999), lr=GENERATOR_LEARNING_RATE) So it's only updating the generator parameters and not the discriminator parameters.

@Saitomar

2 жыл бұрын

I see. Different optimizers are updating different set of model parameters. What is the reasoning behind detaching fake images for discriminator loss? I was thinking that the loss backward will call the computation graph for generator as well, so it's better to detach it from the generator. But since we are only calling the optimizer for discriminator should it even matter whether we detach the fake image tensor or not ?