Help Privacy Policy Disclaimer
  Advanced SearchBrowse





Estimating Mutual Information via Geodesic kNN


Marx,  Alexander
Databases and Information Systems, MPI for Informatics, Max Planck Society;


Fischer,  Jonas
Databases and Information Systems, MPI for Informatics, Max Planck Society;

External Resource
No external resources are shared
Fulltext (restricted access)
There are currently no full texts shared for your IP range.
Fulltext (public)

(Preprint), 770KB

Supplementary Material (public)
There is no public supplementary material available

Marx, A., & Fischer, J. (2021). Estimating Mutual Information via Geodesic kNN. Retrieved from https://arxiv.org/abs/2110.13883.

Cite as: https://hdl.handle.net/21.11116/0000-0009-B130-8
Estimating mutual information (MI) between two continuous random variables
$X$ and $Y$ allows to capture non-linear dependencies between them,
non-parametrically. As such, MI estimation lies at the core of many data
science applications. Yet, robustly estimating MI for high-dimensional $X$ and
$Y$ is still an open research question.
In this paper, we formulate this problem through the lens of manifold
learning. That is, we leverage the common assumption that the information of
$X$ and $Y$ is captured by a low-dimensional manifold embedded in the observed
high-dimensional space and transfer it to MI estimation. As an extension to
state-of-the-art $k$NN estimators, we propose to determine the $k$-nearest
neighbours via geodesic distances on this manifold rather than form the ambient
space, which allows us to estimate MI even in the high-dimensional setting. An
empirical evaluation of our method, G-KSG, against the state-of-the-art shows
that it yields good estimations of the MI in classical benchmark, and manifold
tasks, even for high dimensional datasets, which none of the existing methods
can provide.