Generative Adversarial Networks

The autoencoder learned to compress and redraw data it had seen. A GAN goes further: it learns to draw data that never existed — faces, digits, artwork — by turning generation into a game between two networks. In this final lesson you'll build the adversarial intuition with a demo small enough to run in your browser, then scale the same idea up to a DCGAN that generates handwritten digits.

The counterfeiter and the detective

A GAN (generative adversarial network, Goodfellow et al., 2014) trains two networks against each other:

• The generator G is a counterfeiter. It takes a random noise vector z and transforms it into a fake sample, G(z) — a fake image, say. It never sees real data directly.
• The discriminator D is a detective. Given a sample, it outputs the probability that the sample is real. It trains on both real data (label 1) and the generator's fakes (label 0).

They improve because of each other. Early on, the fakes are garbage and the detective wins easily. But the detective's verdicts flow back through backpropagation as a training signal for the counterfeiter: "this fake was spotted because of these pixels." The counterfeiter adjusts, the fakes get better, the detective is forced to sharpen its criteria, and around it goes. At the theoretical equilibrium, the fakes are indistinguishable from real data and the detective is reduced to guessing — 50/50.

Formally the game is a minimax objective: min_G max_D E[log D(x)] + E[log(1 − D(G(z)))]. In plain words: the discriminator tunes its weights to maximize its accuracy — assign high D(x) to real samples and low D(G(z)) to fakes — while the generator tunes its weights to minimize that same score by making fakes the discriminator scores high. One value function, two players pulling in opposite directions.

The adversarial game in 1-D, live

Images are too big to watch a GAN think, so let's shrink the problem until every moving part is visible. The "real data" is just numbers drawn from N(4, 0.5). The generator is the simplest possible one — it reshapes standard Gaussian noise as x = mu + sigma * z, so its only weights are mu and sigma. The discriminator is a tiny logistic regression on the features x and x² (quadratic features can perfectly separate two Gaussians). Both are trained with plain gradient steps, alternating — a real GAN loop, in numpy:

The generator starts producing numbers around 0 and, guided only by the discriminator's verdicts, marches its distribution over to sit on top of the real one. Look honestly at the final numbers, though: mu lands near 4 but sigma settles around 0.7 rather than 0.5, and D(fake) oscillates instead of resting at 0.5. Even in one dimension with four parameters, the game circles the equilibrium rather than settling on it — remember this when we discuss instability.

Scaling up: DCGAN on MNIST

For images, both players become convolutional networks — the DCGAN recipe. The generator runs a CNN in reverse: ConvTranspose2d layers upsample a noise vector into an image, doubling resolution at each step. The discriminator is an ordinary CNN classifier ending in a single logit. This needs a GPU — run it in the notebook on Colab.

import torch
from torch import nn
from torch.utils.data import DataLoader
from torchvision import datasets, transforms
 
device = "cuda" if torch.cuda.is_available() else "cpu"
 
# scale images to [-1, 1] to match the generator's Tanh output
tfm = transforms.Compose([transforms.ToTensor(),
                          transforms.Normalize(0.5, 0.5)])
train_set = datasets.MNIST("data", train=True, download=True, transform=tfm)
loader = DataLoader(train_set, batch_size=128, shuffle=True)
 
z_dim = 100
 
generator = nn.Sequential(
    # (batch, 100, 1, 1) -> (batch, 128, 7, 7)
    nn.ConvTranspose2d(z_dim, 128, kernel_size=7, stride=1, padding=0),
    nn.BatchNorm2d(128), nn.ReLU(),
    # -> (batch, 64, 14, 14)
    nn.ConvTranspose2d(128, 64, kernel_size=4, stride=2, padding=1),
    nn.BatchNorm2d(64), nn.ReLU(),
    # -> (batch, 1, 28, 28)
    nn.ConvTranspose2d(64, 1, kernel_size=4, stride=2, padding=1),
    nn.Tanh(),
).to(device)
 
discriminator = nn.Sequential(
    # (batch, 1, 28, 28) -> (batch, 64, 14, 14)
    nn.Conv2d(1, 64, kernel_size=4, stride=2, padding=1),
    nn.LeakyReLU(0.2),
    # -> (batch, 128, 7, 7)
    nn.Conv2d(64, 128, kernel_size=4, stride=2, padding=1),
    nn.BatchNorm2d(128), nn.LeakyReLU(0.2),
    # -> (batch, 1, 1, 1) -> one logit per image
    nn.Conv2d(128, 1, kernel_size=7, stride=1, padding=0),
    nn.Flatten(),
).to(device)

The training loop alternates the two players every batch. Note the .detach() when training the discriminator (no generator gradients wanted) and the flipped labels when training the generator:

criterion = nn.BCEWithLogitsLoss()
opt_d = torch.optim.Adam(discriminator.parameters(), lr=2e-4, betas=(0.5, 0.999))
opt_g = torch.optim.Adam(generator.parameters(), lr=2e-4, betas=(0.5, 0.999))
 
fixed_z = torch.randn(64, z_dim, 1, 1, device=device)  # to watch progress
 
for epoch in range(25):
    for real, _ in loader:
        real = real.to(device)
        bs = len(real)
        ones, zeros = torch.ones(bs, 1, device=device), torch.zeros(bs, 1, device=device)
 
        # --- 1. discriminator step ---
        z = torch.randn(bs, z_dim, 1, 1, device=device)
        fake = generator(z)
        loss_d = (criterion(discriminator(real), ones) +
                  criterion(discriminator(fake.detach()), zeros))
        opt_d.zero_grad(); loss_d.backward(); opt_d.step()
 
        # --- 2. generator step: make D say 'real' on fakes ---
        loss_g = criterion(discriminator(fake), ones)
        opt_g.zero_grad(); loss_g.backward(); opt_g.step()
 
    print(f"epoch {epoch + 1}: loss_D={loss_d.item():.3f} loss_G={loss_g.item():.3f}")
 
    if (epoch + 1) % 5 == 0:            # sample a grid every few epochs
        from torchvision.utils import make_grid
        import matplotlib.pyplot as plt
        with torch.no_grad():
            samples = generator(fixed_z).cpu() * 0.5 + 0.5   # back to [0, 1]
        plt.figure(figsize=(6, 6))
        plt.imshow(make_grid(samples, nrow=8).permute(1, 2, 0), cmap="gray")
        plt.axis("off"); plt.title(f"epoch {epoch + 1}")
        plt.show()

Because fixed_z never changes, the periodic grids show the same 64 noise vectors maturing from static, to blobs, to recognizable digits over about 25 epochs. Every digit in the final grid is a drawing that exists nowhere in MNIST.

Why GAN training is famously unstable

Unlike every loss you've minimized so far, a GAN has no single number going reliably down — it's two losses chasing each other, and the "landscape" moves whenever either player does. The classic failure modes:

• Mode collapse. The generator finds one output that reliably fools the discriminator and produces only that — a GAN that draws convincing 1s and nothing else. Diversity dies because the objective never explicitly demands it. (Our 1-D demo showed a cousin of this: on two-mode data, a too-simple generator drifts to one mode or smears across both.)
• Non-convergence. The players can circle each other forever — generator adapts, discriminator re-adapts, losses oscillate — without approaching equilibrium, exactly like the residual wobble in the 1-D demo.
• Imbalance. A discriminator that wins too hard gives the generator near-zero gradients; one that's too weak gives it meaningless guidance.

The battle-tested tricks, most from the DCGAN paper, and already baked into the code above: learning rate 2e-4 with Adam betas (0.5, 0.999) (the lower momentum term damps the oscillation), BatchNorm in both networks, LeakyReLU in the discriminator, Tanh output with inputs normalized to [-1, 1], and strided convolutions instead of pooling. Later research added better objectives (Wasserstein loss, spectral normalization) attacking the same instabilities.

GANs today

Honest modern context: for image generation, diffusion models — the engines behind Stable Diffusion and friends — have largely displaced GANs. They optimize a plain denoising objective (an idea you already met in the denoising autoencoder!), which sidesteps the two-player instability and covers modes much more reliably. But GANs remain in real use where speed matters — a GAN generates in one forward pass versus a diffusion model's many denoising steps — for super-resolution, image-to-image translation, and as adversarial components: many state-of-the-art systems still bolt on a discriminator as an extra "does this look real?" loss. The adversarial idea outlived the architecture.

That's a wrap — not just on generative models, but on the entire deep learning course. You started with a single neuron and ended by training two networks to out-scheme each other into drawing digits from pure noise: backpropagation, CNNs, transfer learning, RNNs, embeddings, autoencoders, and GANs are all in your toolkit now. Congratulations on finishing! When you're ready for more, head back to the course catalog and pick your next adventure.