JS Divergence and GANs
The original GAN objective can be analyzed through JS divergence. A discriminator learns to distinguish real samples from generated samples. At its optimum, it encodes a density-ratio signal between the data distribution and the generator distribution. The generator improves by making those distributions harder to distinguish.
Support Mismatch
When real and generated samples live on nearly disjoint supports, JS can saturate. The discriminator becomes very confident, and the generator may receive weak or unhelpful gradients. This is one motivation for Wasserstein GANs, which replace the divergence with an earth-mover-style distance that still changes smoothly as distributions move closer.
Practical Reading
JS is useful for understanding distribution distinguishability. If a classifier can easily tell samples from two sources apart, their JS-style separation is large. If it cannot, the distributions are close under that adversarial view.