Title: The Representation Jensen-Shannon Divergence

Abstract: Quantifying the difference between probability distributions is a fundamental problem in machine learning. However, since the underlying distributions of the data are unknown, statistical divergences must be estimated from empirical data. In this work, we propose the representation Jensen-Shannon divergence (RJSD) divergence which avoids estimating the probability density functions by embedding the data in a reproducing kernel Hilbert space (RKHS) where data distributions are represented via uncentered covariance operators. We provide estimators based on Gram matrices and empirical covariance matrices using random Fourier features. Theoretical analysis reveals that RJSD is a lower bound on the Jensen-Shannon divergence, enabling variational estimation. Additionally, we show that RJSD is a higher-order extension of the maximum mean discrepancy (MMD), providing a more sensitive measure of distributional differences. Our experimental results demonstrate RJSD's superiority in two-sample testing, distribution shift detection, and unsupervised domain adaptation, outperforming state-of-the-art techniques. RJSD's versatility and effectiveness make it a promising tool for machine learning research and applications.