The Helmholtz Machine Through Time

Hinton, GE; Dayan, P; To, A; Neal, RM

Local TagsRelease HistoryDetailsSummary

The Helmholtz Machine Through Time

Hinton, G., Dayan, P., To, A., & Neal, R. (1995). The Helmholtz Machine Through Time. In F. Fogelman-Soulie, & R. Gallinari (Eds.), Conférence Internationale sur les Réseaux de Neurones Artificiels (ICANN 1995) (pp. 483-490). Paris, France: RC2.

Item is Released

show all hide all

Basic

show hide

Item Permalink: https://hdl.handle.net/21.11116/0000-0007-558F-9 Version Permalink: https://hdl.handle.net/21.11116/0000-0007-5599-D

Genre: Conference Paper

Files

show Files

Locators

show

hide

Locator:
https://www.cs.toronto.edu/~hinton/absps/hmtt.pdf (Any fulltext) Open Access status unknown

Description:
-

OA-Status:

Creators

show

hide

Creators:
Hinton, GE, Author
Dayan, P¹, Author
To, A, Author
Neal, RM, Author

Affiliations:
1External Organizations, ou_persistent22

Content

show

hide

Free keywords: -

Abstract: We describe the "wake-sleep'' algorithm that allows a multilayer, unsupervised, stochastic neural network to build a hierarchical, top-down generative model of an ensemble of data vectors. Because the generative model uses distributed representations that are a non-linear function of the input, it is intractable to compute the posterior probability distribution over hidden representations given the generative model and the current data vector. It is therefore intractable to fit the generative model to data using standard techniques such as gradient descent or EM. Instead of computing the posterior distribution exactly, a "Helmholtz Machine'' uses a separate set of bottom-up "recognition'' connections to produce a compact approximation to the posterior distribution. The wake-sleep algorithm uses the top-down generative connections to provide training data for the bottom-up recognition connections and vice versa. In this paper, we show that the wake-sleep algorithm can be generalized to model the temporal structure in sequences of data vectors. This gives a very simple online algorithm that fits generative models which have distributed hidden representations which can be exponentially more powerful than conventional Hidden Markov Models.

Details

show

hide

Language(s):

Dates: Date issued: 1995-10

Publication Status: Issued

Pages: -

Publishing info: -

Table of Contents: -

Rev. Type: -

Identifiers: -

Degree: -

Event

show

hide

Title: Conférence Internationale sur les Réseaux de Neurones Artificiels (ICANN 1995)

Place of Event: Paris, France

Start-/End Date: 1995-10-09 - 1995-10-13

Legal Case

show

Project information

show

Source 1

show

hide

Title: Conférence Internationale sur les Réseaux de Neurones Artificiels (ICANN 1995)

Source Genre: Proceedings

Creator(s):
Fogelman-Soulie, F, Editor
Gallinari, R, Editor

Affiliations:
-

Publ. Info: Paris, France : RC2

Pages: - Volume / Issue: - Sequence Number: - Start / End Page: 483 - 490 Identifier: ISBN: 2-910085-18-X