JS Divergence Properties and Uses
JS divergence is popular when a symmetric, bounded distribution comparison is easier to reason about than raw KL. With base-2 logarithms it lies between 0 and 1 bit, and the square root of JS is a true metric.
Use Cases
- Distribution comparison: compare histograms or class-probability profiles.
- Drift monitoring: detect changes between training and serving distributions.
- Clustering distributions: use the square-root metric version for distance-based methods.
- GAN intuition: understand the original adversarial objective.
Limits
JS still depends on binning or density estimation for continuous data. It can also saturate when supports do not overlap, making it less informative about how far distributions must move to match. Earth Mover's Distance addresses that geometric question directly.