Item

Abstract

Kernel canonical correlation analysis (KCCA) is a dimensionality reduction technique for paired data. By finding directions that
maximize correlation, KCCA learns representations that are more closely
tied to the underlying semantics of the data rather than noise. However,
meaningful directions are not only those that have high correlation to another
modality, but also those that capture the manifold structure of the
data. We propose a method that is simultaneously able to find highly
correlated directions that are also located on high variance directions
along the data manifold. This is achieved by the use of semi-supervised
Laplacian regularization of KCCA. We show experimentally that Laplacian
regularized training improves class separation over KCCA with only
Tikhonov regularization, while causing no degradation in the correlation
between modalities. We propose a model selection criterion based on
the Hilbert-Schmidt norm of the semi-supervised Laplacian regularized
cross-covariance operator, which we compute in closed form.