English
 
Help Privacy Policy Disclaimer
  Advanced SearchBrowse

Item

ITEM ACTIONSEXPORT

Released

Paper

Estimating Mutual Information via Geodesic kNN

MPS-Authors
/persons/resource/persons206670

Marx,  Alexander
Databases and Information Systems, MPI for Informatics, Max Planck Society;

/persons/resource/persons229482

Fischer,  Jonas
Databases and Information Systems, MPI for Informatics, Max Planck Society;

External Resource
No external resources are shared
Fulltext (restricted access)
There are currently no full texts shared for your IP range.
Fulltext (public)

arXiv:2110.13883.pdf
(Preprint), 770KB

Supplementary Material (public)
There is no public supplementary material available
Citation

Marx, A., & Fischer, J. (2021). Estimating Mutual Information via Geodesic kNN. Retrieved from https://arxiv.org/abs/2110.13883.


Cite as: https://hdl.handle.net/21.11116/0000-0009-B130-8
Abstract
Estimating mutual information (MI) between two continuous random variables
$X$ and $Y$ allows to capture non-linear dependencies between them,
non-parametrically. As such, MI estimation lies at the core of many data
science applications. Yet, robustly estimating MI for high-dimensional $X$ and
$Y$ is still an open research question.
In this paper, we formulate this problem through the lens of manifold
learning. That is, we leverage the common assumption that the information of
$X$ and $Y$ is captured by a low-dimensional manifold embedded in the observed
high-dimensional space and transfer it to MI estimation. As an extension to
state-of-the-art $k$NN estimators, we propose to determine the $k$-nearest
neighbours via geodesic distances on this manifold rather than form the ambient
space, which allows us to estimate MI even in the high-dimensional setting. An
empirical evaluation of our method, G-KSG, against the state-of-the-art shows
that it yields good estimations of the MI in classical benchmark, and manifold
tasks, even for high dimensional datasets, which none of the existing methods
can provide.