A generative adversarial network (GAN) is a pair of neural networks trained in competition. The generator turns random noise into fake samples, such as images; the discriminator tries to tell those fakes from real data. Each improves by trying to beat the other, until the generator produces samples realistic enough that the discriminator can't reliably tell the difference. GANs produced the first strikingly realistic synthetic faces and still power tasks like super-resolution, although diffusion models now lead text-to-image generation.

Where did GANs come from?

Ian Goodfellow and colleagues introduced GANs in 2014. The idea was elegantly simple: instead of hand-designing a measure of how "realistic" an image is, train a second network to learn that measure, and use it to train the generator. Within a few years, GAN variants were producing high-resolution faces of people who don't exist, and the technique became one of the most influential ideas in deep learning.

How do the generator and discriminator work?

NetworkInputOutputGoal
GeneratorRandom noise vectorA fake sample (for example, an image)Make the discriminator label fakes as real
DiscriminatorA real or fake sampleProbability it is realLabel real as real and fake as fake

An analogy: a counterfeiter (generator) and a bank inspector (discriminator). The inspector learns the tell-tale signs of fakes; the counterfeiter learns from being caught. Over time both get better, and the counterfeits become very convincing.

How does GAN training proceed?

  1. Sample a batch of real data and a batch of noise vectors.
  2. The generator turns the noise into fake samples.
  3. Train the discriminator one step to classify real as 1 and fake as 0.
  4. Train the generator one step to make the discriminator output 1 on its fakes, using gradients that flow back through the discriminator into the generator.
  5. Repeat for many thousands of steps.

Mathematically this is a two-player minimax game. In theory, at equilibrium the generator reproduces the real data distribution and the discriminator outputs 0.5 for everything. In practice, reaching and holding that balance is the hard part.

Why are GANs hard to train?

  • Mode collapse: the generator finds a few outputs that fool the discriminator and keeps producing them, losing diversity. Generating only one style of face is a typical symptom.
  • Vanishing generator gradients: if the discriminator becomes too good too fast, its outputs saturate and the generator learns almost nothing.
  • Oscillation: the two networks chase each other without converging.
  • Hard to evaluate: the losses don't tell you directly whether images are improving.

Many fixes emerged: Wasserstein GANs with gradient penalties for smoother training signals, spectral normalisation, careful architecture choices, label smoothing and progressive growing from low to high resolution.

Which GAN variants are worth knowing?

VariantKey ideaKnown for
DCGANConvolutional generator and discriminator with stable design rulesThe standard starting point
Conditional GANBoth networks also receive a labelGenerating a chosen class (a specific digit)
Pix2PixPaired image-to-image translationSketches to photos, maps to aerial views
CycleGANUnpaired translation with cycle consistencyHorses to zebras, summer to winter
WGAN-GPWasserstein distance with gradient penaltyMore stable training
StyleGANStyle-based generator with fine controlPhotorealistic faces
SRGAN / ESRGANAdversarial loss for super-resolutionSharper upscaled images

How do GANs compare with diffusion models?

AspectGANsDiffusion models
Generation speedOne forward pass: very fastMany denoising steps (faster with modern samplers)
Training stabilityTricky (adversarial balance)Stable (simple denoising loss)
DiversityRisk of mode collapseCovers the data distribution well
Text-to-image qualityGood in narrow domainsState of the art for general prompts

Diffusion's stability and diversity explain why it took over general text-to-image generation. GANs' speed keeps them attractive for real-time tasks and as components inside larger systems. See what is generative AI for how these model families fit together.

What are GANs used for?

  • Super-resolution and restoration: sharpening low-resolution or old photos.
  • Data augmentation: generating extra training examples for rare classes, with careful validation.
  • Image-to-image translation: sketches to renders, day to night, style transfer.
  • Synthetic data for privacy: sharing realistic but artificial records, with strict checks that real individuals can't be recovered.
  • Design exploration: generating variations of patterns, textures or products.

What ethical issues do GANs raise?

Realistic synthetic faces and media can be misused for impersonation and misinformation. Responsible use means consent for training images, clear labelling of synthetic content, refusing harmful applications, and following platform rules and Indian law on deepfakes and impersonation. Detection research and provenance tools such as content credentials aim to help, but responsible practice by builders matters most.

How do you evaluate a GAN?

Because the losses don't directly show quality, practitioners use separate measures. The Fréchet Inception Distance (FID) compares statistics of real and generated images computed by a pretrained network; lower is better. Precision and recall for generative models separate realism (are samples plausible?) from coverage (do they span the variety of real data?). Human evaluation remains important, especially for subtle artefacts. Always inspect many samples visually, including random ones rather than only your favourites, so you notice mode collapse early.

What does the GAN loss look like?

In the original formulation, the discriminator D maximises log D(real) + log(1 − D(fake)): it wants high scores for real samples and low scores for fakes. The generator G tries to minimise log(1 − D(G(z))), meaning it wants its fakes scored as real. Early in training, when fakes are obviously fake, that term gives the generator weak gradients, so practitioners use the "non-saturating" version instead: the generator maximises log D(G(z)). This small change, explained in the original paper, makes early training much more effective, and it's a good example of how practical tricks matter as much as the headline idea.

What is the latent space, and why is it interesting?

The generator's input noise vector lives in a latent space. A well-trained GAN maps nearby points in that space to similar outputs, so moving smoothly between two points morphs one generated face into another. Researchers found directions in latent space that correspond to meaningful changes, such as adding glasses or changing age or pose, which let people edit generated images by moving along those directions. Latent spaces are a recurring idea in generative AI: diffusion models and language models also work with learned internal representations that capture meaningful structure.

What are common mistakes when training a first GAN?

  • Unbalanced networks: a discriminator far stronger than the generator stalls learning.
  • Learning rates that are too high, causing wild oscillation.
  • Judging progress by loss values alone instead of looking at samples.
  • Unnormalised images: scale pixel values to the range the generator's output activation produces.

Start from a known-good DCGAN configuration on a standard dataset, confirm it works, and only then change the data or architecture one step at a time. Saving a grid of samples every few hundred steps makes progress, and problems such as mode collapse, obvious at a glance.

How should you learn GANs?

Build a DCGAN on handwritten digits or small images in PyTorch: watch samples improve over epochs, deliberately make the discriminator too strong and observe the generator stall, then add a conditional label to control which digit is generated. GANs and transfer learning are part of Program Zero's deep learning course, after CNNs and before transformers and LLMs in the generative AI course.