# Item

ITEM ACTIONSEXPORT

Released

Conference Paper

#### Infinite dimensional exponential families by reproducing kernel Hilbert spaces

##### MPS-Authors

There are no MPG-Authors in the publication available

##### External Resource

##### Fulltext (restricted access)

There are currently no full texts shared for your IP range.

##### Fulltext (public)

IGAIA-2005-Fukumizu.pdf

(Any fulltext), 196KB

##### Supplementary Material (public)

There is no public supplementary material available

##### Citation

Fukumizu, K. (2005). Infinite dimensional exponential families by reproducing kernel
Hilbert spaces. In *2nd International Symposium on Information Geometry and its Applications (IGAIA
2005)* (pp. 324-333).

Cite as: https://hdl.handle.net/11858/00-001M-0000-0013-D373-C

##### Abstract

The purpose of this paper is to propose a method of constructing exponential

families of Hilbert manifold, on which estimation theory can be built. Although

there have been works on infinite dimensional exponential families of Banach manifolds

(Pistone and Sempi, 1995; Gibilisco and Pistone, 1998; Pistone and Rogantin,

1999), they are not appropriate to discuss statistical estimation with finite number

of samples; the likelihood function with finite samples is not continuous on the

manifold.

In this paper we use a reproducing kernel Hilbert space as a functional space for

constructing an exponential manifold. A reproducing kernel Hilbert space is dened as a Hilbert space of functions such that evaluation of a function at an arbitrary

point is a continuous functional on the Hilbert space. Since we can discuss the

value of a function with this space, it is very natural to use a manifold associated

with a reproducing kernel Hilbert space as a basis of estimation theory.

We focus on the maximum likelihood estimation (MLE) with the exponential

manifold of a reproducing kernel Hilbert space. As in many non-parametric estimation

methods, straightforward extension of MLE to an infinite dimensional

exponential manifold suffers the problem of ill-posedness caused by the fact that

the estimator should be chosen from the infinite dimensional space with only finite

number of constraints given by the data. To solve this problem, a pseudo-maximum

likelihood method is proposed by restricting the infinite dimensional manifold to

a series of finite dimensional submanifolds, which enlarge as the number of samples

increases. Some asymptotic results in the limit of infinite samples are shown,

including the consistency of the pseudo-MLE.

families of Hilbert manifold, on which estimation theory can be built. Although

there have been works on infinite dimensional exponential families of Banach manifolds

(Pistone and Sempi, 1995; Gibilisco and Pistone, 1998; Pistone and Rogantin,

1999), they are not appropriate to discuss statistical estimation with finite number

of samples; the likelihood function with finite samples is not continuous on the

manifold.

In this paper we use a reproducing kernel Hilbert space as a functional space for

constructing an exponential manifold. A reproducing kernel Hilbert space is dened as a Hilbert space of functions such that evaluation of a function at an arbitrary

point is a continuous functional on the Hilbert space. Since we can discuss the

value of a function with this space, it is very natural to use a manifold associated

with a reproducing kernel Hilbert space as a basis of estimation theory.

We focus on the maximum likelihood estimation (MLE) with the exponential

manifold of a reproducing kernel Hilbert space. As in many non-parametric estimation

methods, straightforward extension of MLE to an infinite dimensional

exponential manifold suffers the problem of ill-posedness caused by the fact that

the estimator should be chosen from the infinite dimensional space with only finite

number of constraints given by the data. To solve this problem, a pseudo-maximum

likelihood method is proposed by restricting the infinite dimensional manifold to

a series of finite dimensional submanifolds, which enlarge as the number of samples

increases. Some asymptotic results in the limit of infinite samples are shown,

including the consistency of the pseudo-MLE.