How do Generative Adversarial Networks (GANs) generate data?
Generative adversarial networks (GANs) learn to generate synthetic data by training two competing neural networks: a generator maps random noise to fake samples, while a discriminator classifies samples as real or fake. The generator gradually learns to fool the discriminator. GANs can produce sharp outputs, but adversarial training is unstable and prone to mode collapse.
Read full answerHide full answer
Applies toPyTorch 2.x (Python 3.12)
How GANs Work
GAN training resembles a two-player minimax game:
-
The generator
Gmaps a latent noise vectorzto a synthetic sample,G(z). -
The discriminator
Dreceives either a real samplexor a generated sample and predicts whether it is real. -
The discriminator learns to classify real samples as real and generated samples as fake.
-
The generator learns to produce samples that the discriminator classifies as real.
-
Each network is updated while the other network’s parameters are temporarily fixed.
Unlike a supervised model, a GAN receives no explicit target outputs. Its learning signal comes from the discriminator’s changing judgments.
Original and Practical Objectives
The original minimax objective is:
min_G max_D ( E[x ~ p_data][log D(x)] + E[z ~ p_noise][log(1 - D(G(z)))] )
-
GandDare the generator and discriminator. -
D(x)is the discriminator’s probability that real samplexis real. -
G(z)is the sample generated from latent vectorz. -
x ~ p_dataandz ~ p_noisedenote samples from the real-data and noise distributions. -
Edenotes an expected value, andlogis the natural logarithm. -
min_G max_Dmeans the discriminator maximizes the objective while the generator minimizes it.
The discriminator minimizes a classification loss, while the generator tries to make its samples appear real. In practice, the non-saturating generator loss is commonly used:
L_G = -E[log D(G(z))]
Minimizing L_G encourages D(G(z)) toward 1 and avoids the saturating gradients associated with the original generator term. With logits, this loss is commonly implemented using BCEWithLogitsLoss.
Training Difficulties, Evaluation, and Ethics
-
Mode collapse: The generator produces only a narrow range of outputs.
-
Vanishing gradients: An overly strong discriminator gives the generator little useful signal.
-
Instability: Loss values oscillate and generated quality can change abruptly.
-
Poor balancing: Learning rates or update frequencies may favor one network.
Remedies include WGAN objectives, WGAN-GP with a gradient penalty, spectral normalization, architecture-appropriate normalization, careful learning rates, and two-timescale updates that give the discriminator or critic more frequent updates. Generated samples must be monitored rather than relying only on loss values.
-
FID measures distance between real and generated feature distributions. Lower values usually indicate better similarity, but results depend on the feature network, sample count, and preprocessing.
-
Inception Score combines classifier confidence with sample diversity. It does not directly measure similarity to the real training distribution and is implementation-dependent.
-
Visual inspection remains important for detecting artifacts, memorization, and diversity problems.
Ethical risks include deepfakes, unauthorized likenesses, misleading media, and unclear training-data provenance. Deployments should use consent, provenance controls, watermarking where appropriate, and safeguards against harmful content.
Architectures and Alternatives
| Variant | Main idea | Typical use |
|---|---|---|
| DCGAN | Uses convolutional generator and discriminator networks | General image generation |
| Conditional GAN | Conditions both networks on a class or attribute | Class-controlled generation |
| Pix2Pix | Learns paired input-output translations | Paired image-to-image tasks |
| CycleGAN | Uses cycle-consistency loss for unpaired translation | Conversion between image domains |
| StyleGAN | Uses layered style parameters to control synthesis | Controllable, high-resolution images |
| Model | How it works | Strengths | Weaknesses | Best fit | Typical scale |
|---|---|---|---|---|---|
| GAN | Adversarial generator-discriminator training | Sharp results and one-pass generation | Unstable; mode collapse | Specialized or real-time generation | One generator pass at inference; training scales with data and model size |
| VAE | Encodes data into a latent distribution and reconstructs it | Stable, likelihood-based training | Often blurrier results | Compact latent representations | Encoder and decoder passes; often smaller models |
| Diffusion model | Iteratively denoises random noise | High quality and comparatively stable training | Many sampling steps and higher latency | General high-quality generation | Many network passes during sampling; compute-heavy training |
Diffusion models are now a main approach for general high-quality image generation. GANs remain useful when single-pass generation, architectural control, or real-time generation matters.
Code Example
import numpy as np
import torch
from torch import nn
torch.manual_seed(42)
rng = np.random.default_rng(42)
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
class Generator(nn.Module):
def __init__(self):
super().__init__()
self.net = nn.Sequential(
nn.Linear(2, 16),
nn.ReLU(),
nn.Linear(16, 1),
)
def forward(self, z):
return self.net(z)
class Discriminator(nn.Module):
def __init__(self):
super().__init__()
self.net = nn.Sequential(
nn.Linear(1, 16),
nn.ReLU(),
nn.Linear(16, 1),
)
def forward(self, x):
return self.net(x)
generator = Generator().to(device)
discriminator = Discriminator().to(device)
optimizer_g = torch.optim.Adam(generator.parameters(), lr=1e-3)
optimizer_d = torch.optim.Adam(discriminator.parameters(), lr=1e-3)
criterion = nn.BCEWithLogitsLoss()
for step in range(1, 3001):
optimizer_d.zero_grad(set_to_none=True)
# Draw real samples from a standard Gaussian distribution.
real_np = rng.standard_normal((128, 1)).astype(np.float32)
real = torch.from_numpy(real_np).to(device)
z = torch.randn(128, 2, device=device)
fake = generator(z).detach() # Do not update G during D training.
loss_d = (
criterion(discriminator(real), torch.ones_like(real))
+ criterion(discriminator(fake), torch.zeros_like(fake))
)
loss_d.backward()
optimizer_d.step()
optimizer_g.zero_grad(set_to_none=True)
z = torch.randn(128, 2, device=device)
fake = generator(z)
loss_g = criterion(discriminator(fake), torch.ones_like(fake))
loss_g.backward()
optimizer_g.step()
if step % 500 == 0:
with torch.no_grad():
sample = generator(torch.randn(1024, 2, device=device))
print(
f"step={step}, d_loss={loss_d.item():.4f}, "
f"g_loss={loss_g.item():.4f}, "
f"sample_mean={sample.mean().item():.3f}, "
f"sample_std={sample.std().item():.3f}"
)
Common Pitfalls and Interview Tips
-
Omitting
detach()lets the discriminator loss update the generator incorrectly. -
Treating low discriminator loss as proof of quality overlooks mode collapse.
-
Reporting only FID or Inception Score hides artifacts and implementation dependencies.
-
An overly strong discriminator can produce vanishing generator gradients.
-
One seed and one checkpoint do not establish robustness; test multiple seeds and checkpoints.
Common follow-up questions concern why BCEWithLogitsLoss is numerically stable, how WGAN-GP improves training, how mode collapse is detected, why FID is imperfect, and why diffusion models have largely replaced GANs for many image-generation tasks.