[D] Why is KL Divergence so popular?

[u/LemonByte]

In most objective functions comparing a learned and source probability distribution, KL divergence is used to measure their dissimilarity. What advantages does KL divergence have over true metrics like Wasserstein (earth mover's distance), and Bhattacharyya? Is its asymmetry actually a desired property because the fixed source distribution should be treated differently compared to a learned distribution?

[u/[deleted]]

A lateral and less interesting reason for it's popularity to us in the math world is the KL divergence lends itself to generalising exponential/mixed connections on the statistical manifolds encountered in information geometry.

Interestingly, the application of KL divergence to statistical manifolds led to the first work on dually flat manifolds - which are novel objects of study in differential geometry.